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Chapter 1 


INTRODUCTION 


1.1 Motivation 


The material in this book originated from attempts to understand and systemize nu- 
merical solution techniques for the partial differential equations governing the physics 
of fluid flow. As time went on and these attempts began to crystallize, underlying 
constraints on the nature of the material began to form. The principal such constraint 
was the demand for unification. Was there one mathematical structure which could 
be used to describe the behavior and results of most numerical methods in common 
use in the field of fluid dynamics? Perhaps the answer is arguable, but the authors 
believe the answer is affirmative and present this book as justification for that be- 
lief. The mathematical structure is the theory of linear algebra and the attendant 
eigenanalysis of linear systems. 

The ultimate goal of the field of computational fluid dynamics (CFD) is to under- 
stand the physical events that occur in the flow of fluids around and within designated 
objects. These events are related to the action and interaction of phenomena such 
as dissipation, diffusion, convection, shock waves, slip surfaces, boundary layers, and 
turbulence. In the field of aerodynamics, all of these phenomena are governed by 
the compressible Navier-Stokes equations. Many of the most important aspects of 
these relations are nonlinear and, as a consequence, often have no analytic solution. 
'This, of course, motivates the numerical solution of the associated partial differential 
equations. At the same time it would seem to invalidate the use of linear algebra for 
the classification of the numerical methods. Experience has shown that such is not 
the case. 

As we shall see in a later chapter, the use of numerical methods to solve partial 
differential equations introduces an approximation that, in effect, can change the 
form of the basic partial differential equations themselves. The new equations, which 
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are the ones actually being solved by the numerical process, are often referred to as 
the modified partial differential equations. Since they are not precisely the same as 
the original equations, they can, and probably will, simulate the physical phenomena 
listed above in ways that are not exactly the same as an exact solution to the basic 
partial differential equation. Mathematically, these differences are usually referred to 
as truncation errors. However, the theory associated with the numerical analysis of 
fluid mechanics was developed predominantly by scientists deeply interested in the 
physics of fluid flow and, as a consequence, these errors are often identified with a 
particular physical phenomenon on which they have a strong effect. Thus methods are 
said to have a lot of "artificial viscosity" or said to be highly dispersive. This means 
that the errors caused by the numerical approximation result in a modified partial 
differential equation having additional terms that can be identified with the physics 
of dissipation in the first case and dispersion in the second. There is nothing wrong, 
of course, with identifying an error with a physical process, nor with deliberately 
directing an error to a specific physical process, as long as the error remains in some 
engineering sense "small". It is safe to say, for example, that most numerical methods 
in practical use for solving the nondissipative Euler equations create a modified partial 
differential equation that produces some form of dissipation. However, if used and 
interpreted properly, these methods give very useful information. 

Regardless of what the numerical errors are called, if their effects are not thor- 
oughly understood and controlled, they can lead to serious difficulties, producing 
answers that represent little, if any, physical reality. This motivates studying the 
concepts of stability, convergence, and consistency. On the other hand, even if the 
errors are kept small enough that they can be neglected (for engineering purposes), 
the resulting simulation can still be of little practical use if inefficient or inappropriate 
algorithms are used. This motivates studying the concepts of stiffness, factorization, 
and algorithm development in general. АП of these concepts we hope to clarify in 
this book. 


1.2 Background 


The field of computational fluid dynamics has a broad range of applicability. Indepen- 
dent of the specific application under study, the following sequence of steps generally 
must be followed in order to obtain a satisfactory solution. 


1.2.1 Problem Specification and Geometry Preparation 


The first step involves the specification of the problem, including the geometry, flow 
conditions, and the requirements of the simulation. The geometry may result from 
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measurements of an existing configuration or may be associated with a design study. 
Alternatively, in a design context, no geometry need be supplied. Instead, a set 
of objectives and constraints must be specified. Flow conditions might include, for 
example, the Reynolds number and Mach number for the flow over an airfoil. The 
requirements of the simulation include issues such as the level of accuracy needed, the 
turnaround time required, and the solution parameters of interest. The first two of 
these requirements are often in conflict and compromise is necessary. As an example 
of solution parameters of interest in computing the flowfield about an airfoil, one may 
be interested in i) the lift and pitching moment only, ii) the drag as well as the lift 
and pitching moment, or iii) the details of the flow at some specific location. 


1.2.2 Selection of Governing Equations and Boundary Con- 
ditions 


Once the problem has been specified, an appropriate set of governing equations and 
boundary conditions must be selected. It is generally accepted that the phenomena of 
importance to the field of continuum fluid dynamics are governed by the conservation 
of mass, momentum, and energy. The partial differential equations resulting from 
these conservation laws are referred to as the Navier-Stokes equations. However, in 
the interest of efficiency, it is always prudent to consider solving simplified forms 
of the Navier-Stokes equations when the simplifications retain the physics which are 
essential to the goals of the simulation. Possible simplified governing equations include 
the potential-How equations, the Euler equations, and the thin-layer Navier-Stokes 
equations. These may be steady or unsteady and compressible or incompressible. 
Boundary types which may be encountered include solid walls, inflow and outflow 
boundaries, periodic boundaries, symmetry boundaries, etc. The boundary conditions 
which must be specified depend upon the governing equations. For example, at a solid 
wall, the Euler equations require flow tangency to be enforced, while the Navier-Stokes 
equations require the no-slip condition. If necessary, physical models must be chosen 
for processes which cannot be simulated within the specified constraints. Turbulence 
is an example of a physical process which is rarely simulated in a practical context (at 
the time of writing) and thus is often modelled. The success of a simulation depends 
greatly on the engineering insight involved in selecting the governing equations and 
physical models based on the problem specification. 


1.2.3 Selection of Gridding Strategy and Numerical Method 


Next a numerical method and a strategy for dividing the flow domain into cells, or 
elements, must be selected. We concern ourselves here only with numerical meth- 
ods requiring such a tessellation of the domain, which is known as a grid, or mesh. 
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Many different gridding strategies exist, including structured, unstructured, hybrid, 
composite, and overlapping grids. Furthermore, the grid can be altered based on 
the solution in an approach known as solution-adaptive gridding. The numerical 
methods generally used in CFD can be classified as finite-difference, finite-volume, 
finite-element, or spectral methods. The choices of a numerical method and a grid- 
ding strategy are strongly interdependent. For example, the use of finite-difference 
methods is typically restricted to structured grids. Here again, the success of a sim- 
ulation can depend on appropriate choices for the problem or class of problems of 
interest. 


1.2.4 Assessment and Interpretation of Results 


Finally, the results of the simulation must be assessed and interpreted. This step can 
require post-processing of the data, for example calculation of forces and moments, 
and can be aided by sophisticated flow visualization tools and error estimation tech- 
niques. It is critical that the magnitude of both numerical and physical-model errors 
be well understood. 


1.3 Overview 


It should be clear that successful simulation of fluid flows can involve a wide range of 
issues from grid generation to turbulence modelling to the applicability of various sim- 
plified forms of the Navier-Stokes equations. Many of these issues are not addressed 
in this book. Some of them are presented in the books by Anderson, Tannehill, and 
Pletcher [1] and Hirsch [2]. Instead we focus on numerical methods, with emphasis 
on finite-difference and finite-volume methods for the Euler and Navier-Stokes equa- 
tions. Rather than presenting the details of the most advanced methods, which are 
still evolving, we present a foundation for developing, analyzing, and understanding 
such methods. 

Fortunately, to develop, analyze, and understand most numerical methods used to 
find solutions for the complete compressible Navier-Stokes equations, we can make use 
of much simpler expressions, the so-called “model” equations. These model equations 
isolate certain aspects of the physics contained in the complete set of equations. Hence 
their numerical solution can illustrate the properties of a given numerical method 
when applied to a more complicated system of equations which governs similar phys- 
ical phenomena. Although the model equations are extremely simple and easy to 
solve, they have been carefully selected to be representative, when used intelligently, 
of difficulties and complexities that arise in realistic two- and three-dimensional fluid 
flow simulations. We believe that a thorough understanding of what happens when 
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numerical approximations are applied to the model equations is a major first step in 
making confident and competent use of numerical approximations to the Euler and 
Navier-Stokes equations. As a word of caution, however, it should be noted that, 
although we can learn a great deal by studying numerical methods as applied to the 
model equations and can use that information in the design and application of nu- 
merical methods to practical problems, there are many aspects of practical problems 
which can only be understood in the context of the complete physical systems. 


1.4 Notation 


The notation is generally explained as it is introduced. Bold type is reserved for real 
physical vectors, such as velocity. The vector symbol” is used for the vectors (or 
column matrices) which contain the values of the dependent variable at the nodes 
of a grid. Otherwise, the use of a vector consisting of a collection of scalars should 
be apparent from the context and is not identified by any special notation. For 
example, the variable u can denote a scalar Cartesian velocity component in the Euler 
and Navier-Stokes equations, a scalar quantity in the linear convection and diffusion 
equations, and a vector consisting of a collection of scalars in our presentation of 
hyperbolic systems. Some of the abbreviations used throughout the text are listed 
and defined below. 


PDE Partial differential equation 
ODE Ordinary differential equation 


OAE Ordinary difference equation 
RHS Right-hand side 


P.S. Particular solution of an ODE or system of ODE’s 
S.S. Fixed (time-invariant) steady-state solution 

k-D k-dimensional space 

(bc) Boundary conditions, usually а vector 


O(a) А term of order (i.e., proportional to) a 
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Chapter 2 


CONSERVATION LAWS AND 
THE MODEL EQUATIONS 


We start out by casting our equations in the most general form, the integral conserva- 
tion-law form, which is useful in understanding the concepts involved in finite-volume 
schemes. The equations are then recast into divergence form, which is natural for 
finite-difference schemes. The Euler and Navier-Stokes equations are briefly discussed 
in this Chapter. The main focus, though, will be on representative model equations, 
in particular, the convection and diffusion equations. These equations contain many 
of the salient mathematical and physical features of the full Navier-Stokes equations. 
The concepts of convection and diffusion are prevalent in our development of nu- 
merical methods for computational fluid dynamics, and the recurring use of these 
model equations allows us to develop a consistent framework of analysis for consis- 
tency, accuracy, stability, and convergence. The model equations we study have two 
properties in common. They are linear partial differential equations (PDE’s) with 
coefficients that are constant in both space and time, and they represent phenomena 
of importance to the analysis of certain aspects of fluid dynamic problems. 


2.1 Conservation Laws 


Conservation laws, such as the Euler and Navier-Stokes equations and our model 
equations, can be written in the following integral form: 


te t» 
| QdV — f QdV + D $ n.FdSdt = n f PdVdt (2.1) 
V (te) V(ti) t 4S(t) t JV(t) 


In this equation, Q is а vector containing the set of variables which are conserved, 
e.g., mass, momentum, and energy, per unit volume. The equation is a statement of 
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the conservation of these quantities in a finite region of space with volume V(t) and 
surface area S(t) over a finite interval of time t9 — tı. In two dimensions, the region 
of space, or cell, is an area A(t) bounded by a closed contour C(t). The vector n is 
a unit vector normal to the surface pointing outward, F is a set of vectors, or tensor, 
containing the flux of Q per unit area per unit time, and P is the rate of production 
of Q per unit volume per unit time. If all variables are continuous in time, then Eq. 
2.1 can be rewritten as 


a QdV + ф n.Fds = | Рау (2.2) 
dt Jv(t) S(t) V(t) 


Those methods which make various numerical approximations of the integrals in Eqs. 
2.1 and 2.2 and find a solution for Q on that basis are referred to as finite-volume 
methods. Many of the advanced codes written for CFD applications are based on the 
finite-volume concept. 

On the other hand, a partial derivative form of a conservation law can also be 
derived. The divergence form of Eq. 2.2 is obtained by applying Gauss's theorem to 
the flux integral, leading to 

до 


ЭР +V.F=P (2.3) 

where V. is the well-known divergence operator given, in Cartesian coordinates, by 
д д д 

V.= (i—+j—4+k— |. 2.4 

(iz tI T =| 20 


and i,j, апа К are unit vectors in the x, у, and z coordinate directions, respectively. 
Those methods which make various approximations of the derivatives in Eq. 2.3 and 
find a solution for Q on that basis are referred to as finite-difference methods. 


2.2 "The Navier-Stokes and Euler Equations 


The Navier-Stokes equations form a coupled system of nonlinear PDE's describing 
the conservation of mass, momentum and energy for a fluid. For a Newtonian fluid 
in one dimension, they can be written as 


üQ OB 
кыо eco 2.5 
or DE o) 
with 
p pu 0 
Ое | pts E quee |= 2197 (2.6) 


е u(e + p) 2 шид“ + KZ 
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where p is the fluid density, u is the velocity, e is the total energy per unit volume, p is 
the pressure, T is the temperature, u is the coefficient of viscosity, and к is the thermal 
conductivity. The total energy e includes internal energy per unit volume pe (where 
є is the internal energy per unit mass) and kinetic energy per unit volume pu?/2. 
These equations must be supplemented by relations between u and к and the fluid 
state as well as an equation of state, such as the ideal gas law. Details can be found 
in Anderson, Tannehill, and Pletcher [1] and Hirsch [2]. Note that the convective 
fluxes lead to first derivatives in space, while the viscous and heat conduction terms 
involve second derivatives. This form of the equations is called conservation-law or 
conservative form. Non-conservative forms can be obtained by expanding derivatives 
of products using the product rule or by introducing different dependent variables, 
such as и and p. Although non-conservative forms of the equations are analytically 
the same as the above form, they can lead to quite different numerical solutions in 
terms of shock strength and shock speed, for example. Thus the conservative form is 
appropriate for solving flows with features such as shock waves. 

Many flows of engineering interest are steady (time-invariant), or at least may be 
treated as such. For such flows, we are often interested in the steady-state solution of 
the Navier-Stokes equations, with no interest in the transient portion of the solution. 


The steady solution to the one-dimensional Navier-Stokes equations must satisfy 
дЕ 
——=0 2 
Az (2.7) 


If we neglect viscosity and heat conduction, the Euler equations are obtained. In 
two-dimensional Cartesian coordinates, these can be written as 


dQ дЕ ӘР 
а ы = ы) 2.8 
ОЮ ОО Т Өй i 
with 
di p pu ро 
2 
а е аа о И (2.9) 
93 pu ри? pv* + р 
qa e u(e + p) v(e 4 p) 


where u and v are the Cartesian velocity components. Later on we will make use of 
the following form of the Euler equations as well: 
д) , ,0Q | ,0Q — 


Bde ды 2 auch ы : 
3: 2| oe 0 (2.10) 


The matrices A — $0 and В = a are known as the flux Jacobians. The flux vectors 
given above are written in terms of the primitive variables, p, u, v, and p. In order 
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to derive the flux Jacobian matrices, we must first write the flux vectors E and F in 
terms of the conservative variables, q1, do, дз, and q4, as follows: 


E 92 
t 2 2 
Ey (y- Da + 518 — 118 
Е = = (2.11) 
E, qaq y—1 аў q2 qo 
gcc (аав) 
Hh q3 
Е, КО 
F= = | (2.12) 
F; (y- Dg + 524 — 1214 
Е -1 (9293 , di 
! лас р) 


We have assumed that the pressure satisfies p = (y — 1)[e — p(u? + v?)/2] from the 
ideal gas law, where y is the ratio of specific heats, c,/cy. From this it follows that 
the flux Jacobian of E can be written in terms of the conservative variables as 





0 1 0 0 
К a21 (3 — »(&) ge NE) yet 
А= = = (2.13) 
Жы Жж (i) © 
a41 042 043 7() 


where 
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ол Г, 
==) (2) (=) (2.14) 


and in terms of the primitive variables as 


| 
B 
2 
| 
E 
x 
T 1 
PLE 
s 
powered 
w 
+ 
PLN 
>is 
М2 
№ 
4 7X 
Ss 
Sl 
Ll 
| 
WA TN, 
ДЕ 
jo ы 
4 7X 
5 е 
SS 


Q43 


A= (2.15) 
—и0 U u 0 
a41 042 Q43 yu 

where 
zu а= 
а = Tv = i 
ue 

a = (у 1)и(и? + v?) = {ж 
Qs. = 1 Nc б (Зи? + v?) 
аз = (1—wv)uv (2.16) 


Derivation of the two forms of B = OF/0Q is similar. The eigenvalues of the flux 
Jacobian matrices are purely real. This is the defining feature of hyperbolic systems 
of PDE's, which are further discussed in Section 2.5. The homogeneous property of 
the Euler equations is discussed in Appendix C. 

The Navier-Stokes equations include both convective and diffusive fluxes. This 
motivates the choice of our two scalar model equations associated with the physics 
of convection and diffusion. Furthermore, aspects of convective phenomena associ- 
ated with coupled systems of equations such as the Euler equations are important in 
developing numerical methods and boundary conditions. Thus we also study linear 
hyperbolic systems of PDE's. 
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2.3 The Linear Convection Equation 


2.3.1 Differential Form 


The simplest linear model for convection and wave propagation is the linear convection 
equation given by the following PDE: 
Qu Н ди 
os a — 
дї Ox 
Here u(x,t) is a scalar quantity propagating with speed a, a real constant which may 


be positive or negative. The manner in which the boundary conditions are specified 
separates the following two phenomena for which this equation is a model: 


=0 (2.17) 


(1) In one type, the scalar quantity u is given on one boundary, corresponding 
to a wave entering the domain through this “inflow” boundary. No bound- 
ary condition is specified at the opposite side, the “outflow” boundary. This 
is consistent in terms of the well-posedness of a 1%-огіег PDE. Hence the 
wave leaves the domain through the outflow boundary without distortion or 
reflection. This type of phenomenon is referred to, simply, as the convection 
problem. It represents most of the “usual” situations encountered in convect- 
ing systems. Note that the left-hand boundary is the inflow boundary when 
а is positive, while the right-hand boundary is the inflow boundary when a is 
negative. 


(2) In the other type, the flow being simulated is periodic. At any given time, 
what enters on one side of the domain must be the same as that which is 
leaving on the other. This is referred to as the biconvection problem. It is 
the simplest to study and serves to illustrate many of the basic properties of 
numerical methods applied to problems involving convection, without special 
consideration of boundaries. Hence, we pay a great deal of attention to it in 
the initial chapters. 


Now let us consider a situation in which the initial condition is given by u(r,0) = 
ug(r), and the domain is infinite. It is easy to show by substitution that the exact 
solution to the linear convection equation is then 


u(x,t) = uo(x — at) (2.18) 


The initial waveform propagates unaltered with speed |a| to the right if a is positive 
and to the left if a is negative. With periodic boundary conditions, the waveform 
travels through one boundary and reappears at the other boundary, eventually re- 
turning to its initial position. In this case, the process continues forever without any 
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change in the shape of the solution. Preserving the shape of the initial condition 
uo(x) can be a difficult challenge for a numerical method. 


2.3.2 Solution in Wave Space 


We now examine the biconvection problem in more detail. Let the domain be given 
by 0 € z € 2л. We restrict our attention to initial conditions in the form 


u(x, 0) = f(0)e'^* (2.19) 


where f(0) is a complex constant, and к is the wavenumber. In order to satisfy the 
periodic boundary conditions, & must be an integer. It is a measure of the number of 
wavelengths within the domain. With such an initial condition, the solution can be 
written as 

u(x,t) = f(t)ei^* (2.20) 


where the time dependence is contained in the complex function f(t). Substituting 
this solution into the linear convection equation, Eq. 2.17, we find that f(t) satisfies 
the following ordinary differential equation (ODE) 


d 
7 = —iakf (2.21) 
which has the solution | 
f(t) = fe" (2.22) 
Substituting f(t) into Eq. 2.20 gives the following solution 
u(x,t) = fe? = fo)? (2.23) 


where the frequency, w, the wavenumber, к, and the phase speed, a, are related by 
ш = ка (2.24) 


The relation between the frequency and the wavenumber is known as the dispersion 
relation. The linear relation given by Eq. 2.24 is characteristic of wave propagation 
in a nondispersive medium. This means that the phase speed is the same for all 
wavenumbers. As we shall see later, most numerical methods introduce some disper- 
sion; that is, in a simulation, waves with different wavenumbers travel at different 
speeds. 

An arbitrary initial waveform can be produced by summing initial conditions of 
the form of Eq. 2.19. For M modes, one obtains 


u(x, 0) = 2 ToU e ens (2.25) 
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where the wavenumbers are often ordered such that ку < кә < *-- € km. Since the 
wave equation is linear, the solution is obtained by summing solutions of the form of 
Eq. 2.23, giving 


M 
u(d te y Sn 80 (2.26) 
m=1 
Dispersion and dissipation resulting from a numerical approximation will cause the 
shape of the solution to change from that of the original waveform. 


2.4 The Diffusion Equation 


2.4.1 Differential Form 


Diffusive fluxes are associated with molecular motion in a continuum fluid. A simple 
linear model equation for a diffusive process is 


ди ou 

эт "ap (2.27) 
where v is a positive real constant. For example, with u representing the tempera- 
ture, this parabolic PDE governs the diffusion of heat in one dimension. Boundary 
conditions can be periodic, Dirichlet (specified u), Neumann (specified ди/дт), or 
mixed Dirichlet /Neumann. 

In contrast to the linear convection equation, the diffusion equation has a nontrivial 
steady-state solution, which is one that satisfies the governing PDE with the partial 
derivative in time equal to zero. In the case of Eq. 2.27, the steady-state solution 
must satisfy 


—=0 (2.28) 
Therefore, u must vary linearly with x at steady state such that the boundary con- 


ditions are satisfied. Other steady-state solutions are obtained if a source term g(x) 
is added to Eq. 2.27, as follows: 


= =p Iz: = se) (2.29) 


giving a steady state-solution which satisfies 


Oru 
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In two dimensions, the diffusion equation becomes 


Qu u Oru 
aU am t ape ev (2.31) 


where g(x,y) is again a source term. The corresponding steady equation is 
Oru " ru 
Ox? ду 


While Eq. 2.31 is parabolic, Eq. 2.32 is elliptic. The latter is known as the Poisson 
equation for nonzero g, and as Laplace's equation for zero g. 


- g(x,y) =0 (2.32) 


2.4.2 Solution in Wave Space 


We now consider a series solution to Eq. 2.27. Let the domain be given by 0 € z € v 
with boundary conditions u(0) = ua, и(т) = up. It is clear that the steady-state 
solution is given by a linear function which satisfies the boundary conditions, i.e., 
h(x) = ua + (uy — u;)r/n. Let the initial condition be 


u(r,0) = х fa (0) sin kmz + h(x) (2:33) 


where к must be an integer in order to satisfy the boundary conditions. A solution 
of the form 


u(x,t) = >, fa (t) sin kme + h(a) (2.34) 


satisfies the initial and boundary conditions. Substituting this form into Eq. 2.27 
gives the following ODE for fm: 


tlm = —к v fm (2.35) 


and we find Я 
Putt = tae te (2.36) 
Substituting fm(t) into equation 2.34, we obtain 


M 
ulz, t= > fm (0)e т" sin x + h(x) (2.37) 

mel 
The steady-state solution (t — oo) is simply h(x). Eq. 2.37 shows that high wavenum- 


ber components (large Km) of the solution decay more rapidly than low wavenumber 
components, consistent with the physics of diffusion. 
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2.5 Linear Hyperbolic Systems 


The Euler equations, Eq. 2.8, form a hyperbolic system of partial differential equa- 
tions. Other systems of equations governing convection and wave propagation phe- 
nomena, such as the Maxwell equations describing the propagation of electromagnetic 
waves, are also of hyperbolic type. Many aspects of numerical methods for such sys- 
tems can be understood by studying a one-dimensional constant-coefficient linear 
system of the form 


— + А— = 0 (2.38) 
7 
where и = u(x,t) is a vector of length m and А is a real m x m matrix. For 
conservation laws, this equation can also be written in the form 


Ou Of _ 





ж = 0 2.39 
aE” Ox ee) 
where f is the flux vector and A = of is the flux Jacobian matrix. The entries in the 
flux Jacobian are 
д} 
p= 2.40 
a J ди; ( ) 


The flux Jacobian for the Euler equations is derived in Section 2.2. 
Such a system is hyperbolic if A is diagonalizable with real eigenvalues.! Thus 


А = X7'AX (2.41) 


where Л is a diagonal matrix containing the eigenvalues of А, and X is the matrix 
of right eigenvectors. Premultiplying Eq. 2.38 by X^ !, postmultiplying A by the 
product X X ^!, and noting that X and X^! are constants, we obtain 


A 





Ө 1 1 1 
Хи OX WAX XU 
= te yy 2.42 
a дг eae) 
With w = X~!u, this can be rewritten as 
Qu дш 
—— —=0 2.43 
at Ox Ти 


When written in this manner, the equations have been decoupled into m scalar equa- 
tions of the form 


gr "^ Eas 


!See Appendix А for a brief review of some basic relations and definitions from linear algebra. 








=0 (2.44) 
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The elements of w are known as characteristic variables. Each characteristic variable 
satisfies the linear convection equation with the speed given by the corresponding 
eigenvalue of A. 

Based on the above, we see that a hyperbolic system in the form of Eq. 2.38 has a 
solution given by the superposition of waves which can travel in either the positive or 
negative directions and at varying speeds. While the scalar linear convection equation 
is clearly an excellent model equation for hyperbolic systems, we must ensure that 
our numerical methods are appropriate for wave speeds of arbitrary sign and possibly 
widely varying magnitudes. 

The one-dimensional Euler equations can also be diagonalized, leading to three 
equations in the form of the linear convection equation, although they remain non- 
linear, of course. The eigenvalues of the flux Jacobian matrix, or wave speeds, are 
u,u+c, and и — c, where u is the local fluid velocity, and с = \/yp/p is the local 
speed of sound. The speed u is associated with convection of the fluid, while u + с 
and u — c are associated with sound waves. Therefore, in a supersonic flow, where 
|u| > c, all of the wave speeds have the same sign. In a subsonic flow, where |u| < с, 
wave speeds of both positive and negative sign are present, corresponding to the fact 
that sound waves can travel upstream in a subsonic flow. 

The signs of the eigenvalues of the matrix A are also important in determining 
suitable boundary conditions. The characteristic variables each satisfy the linear con- 
vection equation with the wave speed given by the corresponding eigenvalue. There- 
fore, the boundary conditions can be specified accordingly. That is, characteristic 
variables associated with positive eigenvalues can be specified at the left boundary, 
which corresponds to inflow for these variables. Characteristic variables associated 
with negative eigenvalues can be specified at the right boundary, which is the in- 
flow boundary for these variables. While other boundary condition treatments are 
possible, they must be consistent with this approach. 


2.6 Problems 


1. Show that the 1-D Euler equations can be written in terms of the primitive 


variables R = [р, u, p|" as follows: 
OR OR 
—— + М —— = 0 
дї Ox 
where 
u p 0 
М=|0 u р! 
0 yp u 


18 


CHAPTER 2. CONSERVATION LAWS AND THE MODEL EQUATIONS 


Assume an ideal gas, p = (y — 1)(e — pu?/2). 


. Find the eigenvalues and eigenvectors of the matrix M derived in question 1. 


. Derive the flux Jacobian matrix А = ӘЕ/дО for the 1-D Euler equations result- 


ing from the conservative variable formulation (Eq. 2.5). Find its eigenvalues 
and compare with those obtained in question 2. 


. Show that the two matrices M and A derived in questions 1 and 3, respectively, 


are related by a similarity transform. (Hint: make use of the matrix 5 = 


0Q/OR.) 


. Write the 2-D diffusion equation, Eq. 2.31, in the form of Eq. 2.2. 


. Given the initial condition u(z, 0) = sin defined on 0 € x < 27, write it in the 


form of Eq. 2.25, that is, find the necessary values of fm(0). (Hint: use M = 2 
with кү = 1 and кә = —1.) Next consider the same initial condition defined 
only at x = 27/4, j = 0,1,2,3. Find the values of fm(0) required to reproduce 
the initial condition at these discrete points using M = 4 with km = m — 1. 


. Plot the first three basis functions used in constructing the exact solution to 


the diffusion equation in Section 2.4.2. Next consider a solution with boundary 
conditions Ua = иь = 0, and initial conditions from Eq. 2.33 with fm(0) = 1 
for 1 < m € 3, fm(0) = 0 for m > 3. Plot the initial condition on the domain 
0 < z € тп. Plot the solution at t = 1 with v = 1. 


. Write the classical wave equation 0?u/0t? = c?0?u/Ox? as a first-order system, 


i.e., in the form 


au 490 _ 


Ce on” 


where U = [0u/0z,0u/0t]". Find the eigenvalues and eigenvectors of A. 


. The Cauchy-Riemann equations are formed from the coupling of the steady 


compressible continuity (conservation of mass) equation 


Opu  Opv | 
дт ^ Oy ° 


and the vorticity definition 
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where w = 0 for irrotational flow. For isentropic and homenthalpic flow, the 
system is closed by the relation 


eG era)” 


Note that the variables have been nondimensionalized. Combining the two 
PDE's, we have 





where 


zo | (—ри | (—рф 
a= (2), PX 0 ). Ше 
One approach to solving these equations is to add a time-dependent term and 
find the steady solution of the following equation: 


Pd OU uud 4 


0 
Ot Or Oy 


(a) Find the flux Jacobians of f and g with respect to q. 
(b) Determine the eigenvalues of the flux Jacobians. 


(c) Determine the conditions (in terms of p and u) under which the system is 
hyperbolic, i.e., has real eigenvalues. 


(d) Are the above fluxes homogeneous? (See Appendix C.) 
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Chapter 3 


FINITE-DIFFERENCE 
APPROXIMATIONS 


In common with the equations governing unsteady fluid flow, our model equations 
contain partial derivatives with respect to both space and time. One can approximate 
these simultaneously and then solve the resulting difference equations. Alternatively, 
one can approximate the spatial derivatives first, thereby producing a system of or- 
dinary differential equations. The time derivatives are approximated next, leading to 
a time-marching method which produces a set of difference equations which can be 
solved. This is the approach emphasized here. In this chapter, the concept of finite- 
difference approximations to partial derivatives is presented. These can be applied 
either to spatial derivatives or time derivatives. Our emphasis in this Chapter is on 
spatial derivatives; time derivatives are treated in Chapter 6. Strategies for applying 
these finite-difference approximations will be discussed in Chapter 4. 

АП of the material below is presented in a Cartesian system. We emphasize the 
fact that quite general classes of meshes expressed in general curvilinear coordinates 
in physical space can be transformed to a uniform Cartesian mesh with equispaced 
intervals in a so-called computational space, as shown in Figure 3.1. The computational 
space is uniform; all the geometric variation is absorbed into variable coefficients of the 
transformed equations. For this reason, in much of the following accuracy analysis 
we use an equispaced Cartesian system without being unduly restrictive or losing 
practical application. 


3.1 Meshes and Finite-Difference Notation 


The simplest mesh involving both time and space is shown in Figure 3.2. An inspec- 
tion of this figure permits us to define the terms and notation needed to describe 
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Figure 3.1: Physical and computational spaces. 


finite difference approximations. In general, the dependent variables, u, for example, 
are functions of the independent variables t, and x, y, 2. For the first several chapters 
we consider primarily the 1-D case и = u(x,t). When only one variable is denoted, 
dependence on the other is assumed. The mesh index for 2 is always j, and that for 
t is always n. Then on an equispaced grid 


yp = ;-jAx (3.1) 
t һ=пАф=пһҺ (3.2) 


where Az is the spacing in т and At the spacing in t, as shown in Figure 3.2. Note 
that h = At throughout. Later k and l are used for y and z in a similar way. When 
n, j, k, | are used for other purposes (which is sometimes necessary), local context 
should make the meaning obvious. 

The convention for subscript and superscript indexing is as follows: 


u(t4 kh) = u([n+klh) = ик 
u(x + mAz) u( +т]Ах) uus (3.3) 


и(2 + тАх,++ kh) = un 


Notice that when used alone, both the time and space indices appear as a subscript, 
but when used together, time is always a superscript and is usually enclosed with 
parentheses to distinguish it from an exponent. 
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Figure 3.2: Space-time grid arrangement. 


Derivatives are expressed according to the usual conventions. Thus for partial 
derivatives in space or time we use interchangeably 


ди ди o?u 
Ozu — a. дм. — TO Orsu — O22” 


Ar БП еїс. (3.4) 


For the ordinary time derivative in the study of ODE’s we use 


, du 


Ue (3.5) 


In this text subscripts on dependent variables are never used to express derivatives. 
Thus и, will not be used to represent the first derivative of u with respect to т. 

The notation for difference approximations follows the same philosophy but (with 
one exception) it is not unique. By this we mean that the symbol ô is used to represent 
a difference approximation to a derivative such that, for example, 


but the precise nature (and order) of the approximation is not carried in the symbol 
ó. Other ways are used to determine its precise meaning. The one exception is the 
symbol A, which is defined such that 


Ata = teat tns ADS Фууу tj; Aun Шау Uae: etc. (3.7) 


When there is no subscript on At or Az, the spacing is uniform. 
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3.2 Space Derivative Approximations 


А difference approximation can be generated or evaluated by means of a simple Taylor 
series expansion. For example, consider u(x,t) with t fixed. Then, following the 
notation convention given in Eqs. 3.1 to 3.3, = = jAz and u(x + kAz) = u(j Ax + 
kAx) = uj,;. Expanding the latter term about x gives 





Ujk = Uj + cafa) +2 5 (kA)? _ += (kAn)" (09) ^ 69 


Local difference approximations to a given partial derivative can be formed from linear 
combinations of uj and иу for k = +1, £2,--- 
For example, consider the Taylor series expansion for из: 


Qu 1 Q?u 1 oru 
SA (22) ч л (Ss) EET ae (ss) Has 
j | j 


j 





Now subtract u; and divide by Az to obtain 


uju; (Ou) |l Qu ic. 
AS = (Fe) «$e» (да) + (3.10) 


Thus the expression (и; т — и;)/ Ax is a reasonable approximation for (2) ‚ as long as 
j 


Az is small. Next consider the space difference approximation (uj,; — uj—1)/(2Az). 
Expand the terms in the numerator about j and regroup the result to form the 
following equation 


Uj+1 o4 Qu dc 5d ju 1 a (м 
| = A — — |... з 
DAT (iz), pu" (5: n 1202" (229 : oy 


When expressed in this manner, it is clear that the discrete terms on the left side of 
the equation represent a first derivative with a certain amount of error which appears 
on the right side of the equal sign. It is also clear that the error depends on the 
grid spacing to a certain order. The error term containing the grid spacing to the 
lowest power gives the order of the method. From Eq. 3.10, we see that the expression 
(изт — u)/ Ax is a first-order approximation to (8°). Similarly, Eq. 3.11 shows that 





(изт = uj1)/(2Ax) is a second-order approximation to a first derivative. The latter 
is referred to as the three-point centered difference approximation, and one often sees 
the summary result presented in the form 


Qu _ Uj+ — Uj- 2 
Э) = ЕЕ! + O(As) (3.12) 
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3.3  Finite-Difference Operators 


3.3.1 Point Difference Operators 


Perhaps the most common examples of finite-difference formulas are the three-point 
centered-difference approximations for the first and second derivatives:! 


Qu 1 
EJ = gi (ta ша) + О(Аз?) (3.13) 
j 
^u 1 
(=) = Ag sn = 2uj + uj) + O(Az’) (3.14) 
j 


These are the basis for point difference operators since they give an approximation to 
a derivative at one discrete point in a mesh in terms of surrounding points. However, 
neither of these expressions tell us how other points in the mesh are differenced or 
how boundary conditions are enforced. Such additional information requires a more 
sophisticated formulation. 


3.3.2 Matrix Difference Operators 


Consider the relation 
1 
(025); = 5 (uii 7 2иу + Uy-1) (3.15) 


which is a point difference approximation to a second derivative. Now let us derive a 
matrix operator representation for the same approximation. Consider the four point 
mesh with boundary points at a and b shown below. Notice that when we speak of 
“the number of points in a mesh”, we mean the number of interior points excluding 
the boundaries. 


Four point mesh. Ax = т/(М +1) 





Now impose Dirichlet boundary conditions, u(0) = ua, u(t) = up and use the 
centered difference approximation given by Eq. 3.15 at every point in the mesh. We 


1 We will derive the second derivative operator shortly. 
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arrive at the four equations: 


1 
(8.20); = Aga (Ча — 2ш + u2) 
1 
(бутш) = E (ш = 2us + из) 
1 
(05:1). = Agi = 2из + Us) 
1 
pth). = Ag (“з — 2u4 + up) (3.16) 
Writing these equations in the more suggestive form 
(билш), = ( Ua —2u, +te2 y Ax? 
(bau), = ( ш о -2ш +из t (3.17) 
lo gue. = ( ио —2из +u, )/Аз? | 
(hu), = ( из —2u4 +w )/Az? 


it is clear that we can express them in a vector-matrix form, and further, that the 
resulting matrix has a very special form. Introducing 


ui Ua 
5 | %2 Aa d 0 
MESE (во) ао (3.18) 
ид Ub 
and 
—2 1 
1 1—2 1 
A= Ag 1 2©: i (3.19) 
1l — 
we can rewrite Eq. 3.17 as 
д.20 = Aŭ + (bc) (3.20) 


This example illustrates a matrix difference operator. Each line of a matrix dif- 
ference operator is based on a point difference operator, but the point operators used 
from line to line are not necessarily the same. For example, boundary conditions may 
dictate that the lines at or near the bottom or top of the matrix be modified. In the 
extreme case of the matrix difference operator representing a spectral method, none 
of the lines is the same. The matrix operators representing the three-point central- 
difference approximations for a first and second derivative with Dirichlet boundary 
conditions on a four-point mesh are 
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0 1 —2 1 
1 -1 0 1 1 1 —2 1 
T 2AT —1 0 1 Óna Az? T. d ab 
—1 0 1 2 


As a further example, replace the fourth line in Eq. 3.16 by the following point 
operator for a Neumann boundary condition (See Section 3.6.): 


(бы), = 2 (Fe). -2 (uus) (322) 


where the boundary condition is 
Qu 
= = 3.29 
e b quo 


5) 


Then the matrix operator for a three-point central-differencing scheme at interior 
points and a second-order approximation for a Neumann condition on the right is 
given by 


—2 1 
1 1 —2 1 


23. 2/3 


Each of these matrix difference operators is a square matrix with elements that 
are all zeros except for those along bands which are clustered around the central 
diagonal. We call such а matrix a banded matrix and introduce the notation 


b 1 
a 


Sey 


B(M : a,b,c) = (3.25) 


B C Ы 

а ф М 

where use of M in the argument is optional and the illustration is given for a simple 
tridiagonal matrix although any number of bands is a possibility. А tridiagonal matrix 
without constants along the bands can be expressed as B (a, b, c). The arguments for 


a banded matrix are always odd in number and the central one always refers to the 
central diagonal. 
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We can now generalize our previous examples. Defining d as? 


ui 
U2 
ü= | Us (3.26) 
UM 
we can approximate the second derivative of u by 
Õssu = Bü. —2,1)u + (be (3.27) 
x 


where (bc) stands for the vector holding the Dirichlet boundary conditions on the 
left and right sides: 


1 1 
(во) = Agi lta 0, --+,0, us)” (3.28) 


If we prescribe Neumann boundary conditions on the right side, as in Eqs. 3.24 and 
3.22, we find 


^ 1 


бш = св В(а,&,1)и + (bc) (3.29) 
where 

oes TE oes; /3]* 

РЕ m edocet 


Е 1 2Ax (ди\ |" 
6) = aspere (85), 


Notice that the matrix operators given by Eqs. 3.27 and 3.29 carry more informa- 
tion than the point operator given by Eq. 3.15. In Eqs. 3.27 and 3.29, the boundary 
conditions have been uniquely specified and it is clear that the same point operator 
has been applied at every point in the field except at the boundaries. The ability to 
specify in the matrix derivative operator the exact nature of the approximation at the 
various points in the field including the boundaries permits the use of quite general 
constructions which will be useful later in considerations of stability. 


?Note that @ is a function of time only since each element corresponds to one specific spatial 
location. 
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Since we make considerable use of both matrix and point operators, it is important 
to establish a relation between them. A point operator is generally written for some 
derivative at the reference point j in terms of neighboring values of the function. For 
example 


(и); = 02uj-2 + 01Uj—1 + bu; + Сиу +1 (3.30) 


might be the point operator for а first derivative. The corresponding matrix operator 
has for its arguments the coefficients giving the weights to the values of the function 
at the various locations. A j-shift in the point operator corresponds to a diagonal 
shift in the matrix operator. Thus the matrix equivalent of Eq. 3.30 is 


bgt = B(as, a1, b, c4, 0) (3.31) 


Note the addition of a zero in the fifth element which makes it clear that b is the 
coefficient of u;. 


3.3.3 Periodic Matrices 


The above illustrated cases in which the boundary conditions are fixed. If the bound- 
ary conditions are periodic, the form of the matrix operator changes. Consider the 
eight-point periodic mesh shown below. This can either be presented on a linear mesh 
with repeated entries, or more suggestively on a circular mesh as in Figure 3.3. When 
the mesh is laid out on the perimeter of a circle, it doesn’t really matter where the 
numbering starts as long as it “ends” at the point just preceding its starting location. 


1 
0 — 27 — 
0 


E. ve зиз SEO teet ue. 7; 





The matrix that represents differencing schemes for scalar equations on a periodic 
mesh is referred to as a periodic matrix. A typical periodic tridiagonal matrix operator 
with nonuniform entries is given for a 6-point mesh by 


b, C2 G6 
a, b єз 
>>> ао бз c 
B,(6: a,b,c) = Hs b, " (3.32) 
ал bs C6 


Cy a5 be 
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Figure 3.3: Eight points on a circular mesh. 


3.3.4  Circulant Matrices 


In general, as shown in the example, the elements along the diagonals of a periodic 
matrix are not constant. However, a special subset of a periodic matrix is the circulant 
matrix, formed when the elements along the various bands are constant. Circulant 
matrices play a vital role in our analysis. We will have much more to say about them 
later. The most general circulant matrix of order 4 is 


bo bı b2 bs 
бз bo bı b» 
bo b3 bg bı 
b; by з bo 


(3.33) 


Notice that each row of a circulant matrix is shifted (see Figure 3.3) one element to 
the right of the one above it. The special case of a tridiagonal circulant matrix is 
given by 


B,(M : a,b,c) = a (3.34) 
a be : 
с а b| мы 


When the standard three-point central-differencing approximations for a first and 
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second derivative, see Eq. 3.21, are used with periodic boundary conditions, they take 
the form 


0 1 =1 
ж ss di 
Mp EC = — B,(-1,0,1 
Om олт] і 0 Ж ылу ^ 590 
1 -1 
and 
-2 1 1 
1 j2 3 1 
(92), = Aa 1-2 1 | = 92800-21) (3.35) 
1 MS 


Clearly, these special cases of periodic operators are also circulant operators. Later 
on we take advantage of this special property. Notice that there are no boundary 
condition vectors since this information is all interior to the matrices themselves. 


3.4 Constructing Differencing Schemes of Any Or- 
der 


3.4.1 Taylor Tables 


The Taylor series expansion of functions about a fixed point provides a means for 
constructing finite-difference point-operators of any order. A simple and straight 
forward way to carry this out is to construct a “Taylor table,” which makes extensive 
use of the expansion given by Eq. 3.8. As an example, consider Table 3.1 which 
represents a Taylor table for an approximation of a second derivative using three 
values of the function centered about the point at which the derivative is to be 
evaluated. 
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д?и 1 
(Z) Es Age (i + buj + сиу) =? 
j 


Ag: (2), 


Ox? 

а" Uj-1 
b- Uj 

C> Uj+1 


Table 3.1. Taylor table for centered 3-point Lagrangian approximation to a second 
derivative. 





The table is constructed so that some of the algebra is simplified. At the top 
of the table we see an expression with a question mark. This represents one of the 
questions that a study of this table can answer; namely, what is the local error caused 
by the use of this approximation? Notice that all of the terms in the equation appear 
in a column at the left of the table (although, in this case, Az? has been multiplied 
into each term which simplifes the terms to be put into the table.). Then notice 
that at the head of each column there appears the common factor that occurs in the 
expansion of each term about the point 7, that is, 


k 
act (22) КОЕ 
T jj 


The columns to the right of the leftmost one, under the headings, make up the Taylor 
table. Each entry is the coefficient of the term at the top of the corresponding column 
in the Taylor series expansion of the term to the left of the corresponding row. For 
example, the last row in the table corresponds to the Taylor series expansion of cu; 4: 


1 д 1 д? 
Cum = сиу+с-(1)- 157 : (2) +e-(1)?- 542 : (=) 
j j 


E URN Cu ТО EN o NN 
+c- (1) gat Ж ave (1) 5g ne mm 2 (3.36) 


A Taylor table is simply a convenient way of forming linear combinations of Taylor 
series on a term by term basis. 
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Consider the sum of each of these columns. To maximize the order of accuracy 
of the method, we proceed from left to right and force, by the proper choice of a, b, 
and c, these sums to be zero. One can easily show that the sums of the first three 
columns are zero if we satisfy the equation 


11 1|{@ 0 
—1 0 р рер 0 
1 0 1| |с c 


The solution is given by |a, b,c] = —[1,—2, 1]. 
The columns that do not sum to zero constitute the error. 


We designate the first nonvanishing sum to be er;, and refer to 
it as the Taylor series error. 


In this case er, occurs at the fifth column in the table (for this example all even 
columns will vanish by symmetry) and one finds 


a с 0*u Az? ( 0*u 

c eA pt etin 3.37 
217 7 Á l 12 (оа), C 
Note that the Az? has been divided through to make the error term consistent. We 


have just derived the familiar 3-point central differencing point operator for a second 
derivative 


Єт: = 





1 
Ах? 


д?и 1 2 
ga | = Agi tia 7 20 + uj) = О(Ат) (3.38) 
j 


The Taylor table for a 3-point backward-differencing operator representing a first 
derivative is shown in Table 3.2. 


Qu 1 
| ) + — (a2 + 01uj—1 + bu;) = 
j 


дт Ax 


Table 3.2. Taylor table for backward 3-point Lagrangian approximation to a first 
derivative. 
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This time the first three columns sum to zero if 


11 1 аә 0 

De рат |e a 

4 1 0 b 0 
which gives [a9, a4, b] = —3(1, —4,3]. In this case the fourth column provides the 

leading truncation error term: 
1 809 a 3 (Ou Ax? [ д?и 

=——|———©——|АлЗ[—| =——|-— 3.39 
аж 6 ad (5: j 3. gs j (3:39) 








Thus we have derived a second-order backward difference approximation of a first 
derivative: 


0 1 
(Fe) 7 gaz i22 T 40-1 + Зиу) = O(Aa?) (3.40) 
2 


3.4.2 Generalization of Difference Formulas 


In general, a difference approximation to the mth derivative at grid point 7 can be 
cast in terms of q 4- p 4- 1 neighboring points as 


m q 
(cx) + У шашуы = ет (3.41) 


дтт 


j =—р 

where the а; are coefficients to be determined through the use of Taylor Tables to 
produce approximations of a given order. Clearly this process can be used to find 
forward, backward, skewed, or central point operators of any order for any derivative. 
It could be computer automated and extended to higher dimensions. More important, 
however, is the fact that it can be further generalized. In order to do this, let us 
approach the subject in a slightly different way, that is from the point of view of 
interpolation formulas. 'These formulas are discussed in many texts on numerical 
analysis. 


3.4.3 Lagrange and Hermite Interpolation Polynomials 


The Lagrangian interpolation polynomial is given by 


u(x) = M а(х), (3.42) 
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where ак(ж) are polynomials іп т of degree K. The construction of the а(х) can be 
taken from the simple Lagrangian formula for quadratic interpolation (or extrapola- 
tion) with non-equispaced points 


T2 — 1T) 
T2 — 71) 
11 — 1) 
11 — 72) 


(z1 — £)(x2 — x) (zo — x) 
° (z1 — £0) (22 — xo) + “Tro — 21) 
) 
) 


u(r)-—u 


(zo = 2 


+U2 


| 
( 
(3.43) 


(20 — T2 


Notice that the coefficient of each ик is one when x = тк, and zero when x takes 
any other discrete value in the set. If we take the first or second derivative of u(x), 
impose an equispaced mesh, and evaluate these derivatives at the appropriate dis- 
crete point, we rederive the finite difference approximations just presented. Finite 
difference schemes that can be derived from Eq. 3.42 are referred to as Lagrangian 
approximations. 

А generalization of the Lagrangian approach is brought about by using Hermitian 
interpolation. To construct a polynomial for u(x), Hermite formulas use values of the 
function and its derivative(s) at given points in space. Our illustration is for the case 
in which discrete values of the function and its first derivative are used, producing 
the expression 


т) = M ак(х)ик + у b. (x) (=) | (3.44) 


Obviously higher-order derivatives could be included as the problems dictate. А com- 
plete discussion of these polynomials can be found in many references on numerical 
methods, but here we need only the concept. 

The previous examples of a Taylor table constructed explicit point difference op- 
erators from Lagrangian interpolation formulas. Consider next the Taylor table for 
an implicit space differencing scheme for a first derivative arising from the use of a 
Hermite interpolation formula. Based on Eq. 3.44, a generalization of Eq. 3.41 can 
include derivatives at neighboring points, i.e., 


5 oru q 
у bi Этт + › QiUj+i = €T, (3.45) 
TOP 


i=-r ї=—р 


analogous to Eq. 3.44. An example formula is illustrated at the top of Table 3.3. Here 
not only is the derivative at point 7 represented, but also included are derivatives at 
points 7 — 1 and 7 +1, which also must be expanded using Taylor Series about point 


36 CHAPTER 3. FINITE-DIFFERENCE APPROXIMATIONS 


j. This requires the following generalization of the Taylor series expansion given in 


Eq. 3.8: 
ae = | [x 2 (EAa)' D а (x) | (3.46) 


The derivative terms now have coefficients (the coefficient on the j point is taken 
as one to simplify the algebra) which must be determined using the Taylor Table 
approach as outlined below. 





1 
+ Az (qui + bu; + сиз) =? 





1 1 1 0 Ота 0 
—1 0 1 1 1|[% -1 
1 0 1 -2 2||с|= |0 
—1 0 1 3 3,||d 0 
1 01 -4 4] Le 0 


having the solution [а, b,c, d,e] = 1[3,0, —3, 1, 1]. Under these conditions the sixth 


column sums to 
Azt (ди 
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and the method can be expressed as 


m) EJ (2) 3 4 
zm TEL deb — (uat шуы) = O(Az") (3.48) 
(2 id Ox j Ox ud Ах 7 d 


This is also referred to as a Padé formula. 


3.4.4 Practical Application of Padé Formulas 


It is one thing to construct methods using the Hermitian concept and quite another 
to implement them in a computer code. In the form of a point operator it is probably 
not evident at first just how Eq. 3.48 can be applied. However, the situation is quite 
easy to comprehend if we express the same method in the form of a matrix operator. 
A banded matrix notation for Eq. 3.48 is 

гр. BR 1)u + (бе) (3.49) 

= u = ——B(- u с : 

6 ? ? т IAG ? ? 
in which Dirichlet boundary conditions have been imposed.? Mathematically this is 
equivalent to 


даи = 6[В(1,4,1)] "|; -B(-1,0,1)u + (ve) (3.50) 
which can be reexpressed by the “predictor-corrector” sequence 
2 1 > > 
й = gALB(CLO Due (èc) 
би = 6[В(1,4,1)[ tă (3.51) 


With respect to practical implementation, the meaning of the predictor in this 
sequence should be clear. It simply says — take the vector array а, difference it, 
add the boundary conditions, and store the result in the intermediate array ü. The 
meaning of the second row is more subtle, since it is demanding the evaluation of an 
inverse operator, but it still can be given a simple interpretation. Ап inverse matrix 
operator implies the solution of a coupled set of linear equations. These operators 
are very common in finite difference applications. They appear in the form of banded 
matrices having a small bandwidth, in this case a tridiagonal. The evaluation of 
[B(1,4,1)| ! is found by means of a tridiagonal “solver”, which is simple to code, 
efficient to run, and widely used. In general, Hermitian or Padé approximations can 
be practical when they can be implemented by the use of efficient banded solvers. 


?In this case the vector containing the boundary conditions would include values of both и and 
Ou/Ox at both boundaries. 


38 CHAPTER 3. FINITE-DIFFERENCE APPROXIMATIONS 


3.4.5 Other Higher-Order Schemes 


Hermitian forms of the second derivative can also be easily derived by means of a 
Taylor table. For example 


беки = 12 [B(1, 10, 1)] ! Fate -2,1)u 4- (ve) (3.52) 


is O(Az*) and makes use of only tridiagonal operations. It should be mentioned that 
the spline approximation is one form of a Padé matrix difference operator. It is given 
by 


Е 1 
X 


1, —2, 1)u + (ve) (3.53) 
but its order of accuracy is only O(Az?). How much this reduction in accuracy is 
offset by the increased global continuity built into a spline fit is not known. We note 
that the spline fit of a first derivative is identical to any of the expressions in Eqs. 
3.48 to 3.51. 

А final word on Hermitian approximations. Clearly they have an advantage over 
3-point Lagrangian schemes because of their increased accuracy. However, a more 
subtle point is that they get this increase in accuracy using information that is still 
local to the point where the derivatives are being evaluated. In application, this can 
be advantageous at boundaries and in the vicinity of steep gradients. It is obvious, of 
course, that five point schemes using Lagrangian approximations can be derived that 
have the same order of accuracy as the methods given in Eqs. 3.48 and 3.52, but they 
will have a wider spread of space indices. In particular, two Lagrangian schemes with 
the same order of accuracy are (here we ignore the problem created by the boundary 
conditions, although this is one of the principal issues in applying these schemes): 


ди 1 P 

x BT —1)u = О(Ах* 54 

3; ^ тот Bell: —8,0,8,—1)ш O(A«*) (3.54) 
Pu 1 2 
L cue Bats те 280,16 = 4 | 
ag? тада P1, 16, 730,16, - D) O(A«*) (3.55) 


3.5 Fourier Error Analysis 


In order to select a finite-difference scheme for a given application one must be able 
to assess the accuracy of the candidate schemes. The accuracy of an operator is often 
expressed in terms of the order of the leading error term determined from a Taylor 
table. While this is a useful measure, it provides a fairly limited description. Further 
information about the error behavior of a finite-difference scheme can be obtained 
using Fourier error analysis. 
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3.5.1 Application to a Spatial Operator 


An arbitrary periodic function can be decomposed into its Fourier components, which 
are in the form e'^*, where к is the wavenumber. It is therefore of interest to examine 
how well a given finite-difference operator approximates derivatives of е. We will 
concentrate, here, on first derivative approximations, although the analysis is equally 
applicable to higher derivatives. 

The exact first derivative of е is 


д gift 
Ox 


If we apply, for example, a second-order centered difference operator to uj = е‘, 
where x; = Ј Ат, we get 





= ike" (3.56) 


Uj+1 — Uj-1 
$ с = ing J 
(ьш) 2Ax 
gi Av G1) = eikAa(j—1) 


2Ахт 
(607 = eater 


2Ax 


1 
= 2A; C08 KA + їзїп Az) — (cos kAx — isin kAx)]e 


sin KAT 
i Аз 
= ike" (3.57) 


IKT 


KT 


where к* is the modified wavenumber. The modified wavenumber is so named because 
it appears where the wavenumber, к, appears in the exact expression. Thus the degree 
to which the modified wavenumber approximates the actual wavenumber is a measure 
of the accuracy of the approximation. 
For the second-order centered difference operator the modified wavenumber is 
given by 
,  SinkAx 


Note that к* approximates к to second-order accuracy, as is to be expected, since 


sin KAT KAT? 





Ax a 6 
Equation 3.58 is plotted in Figure 3.4, along with similar relations for the stan- 
dard fourth-order centered difference scheme and the fourth-order Padé scheme. The 
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Figure 3.4: Modified wavenumber for various schemes. 


expression for the modified wavenumber provides the accuracy with which a given 
wavenumber component of the solution is resolved for the entire wavenumber range 
available in a mesh of a given size, 0 € &Az < m. 

In general, finite-difference operators can be written in the form 


(0:); = (62); + (б); 


where (02); is an antisymmetric operator and (0$); is a symmetric operator. If we 
restrict our interest to schemes extending from j — 3 to 7 + 3, then 


1 
(и); = Fy laa (uia 7 ji) + ар(иу+ә — uj-2) + аз(иу+з — шу—3)] 


апа 


1 
(беи); = Az dou + й\(иў+1 + uj) + do(uj+2 + изә) + аз (изз + 53) 


The corresponding modified wavenumber is 


1 
Ux = A, do + 2(d; cos &Ax + dz cos 2kAx + аз cos 3k&Ax) 
d 
+ 2i(a;sin KAT + a5 sin 2&Az + az sin 3&Ax) (3.59) 


^In terms of a circulant matrix operator A, the antisymmetric part is obtained from (A — А7)/2 
and the symmetric part from (А + AT)/2. 
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When the finite-difference operator is antisymmetric (centered), the modified wavenum- 
ber is purely real. When the operator includes a symmetric component, the modified 


wavenumber is complex, with the imaginary component being entirely error. The 
fourth-order Padé scheme is given by 


3 
(буш)у—1 + 4(0,u); + (и); = Ag sn 55-4) 


'The modified wavenumber for this scheme satisfies? 


| | a us 
inte ‘А + 4ik* + ik! ci^ = x (en _ QE) 
which gives 
X" 3isin KAT 
К == Ore 
(2 + совкАт)Ат 


The modified wavenumber provides a useful tool for assessing difference approx- 
imations. In the context of the linear convection equation, the errors can be given 
a physical interpretation. Consider once again the linear convection equation in the 
form 


on a domain extending from —oo to oo. Recall from Section 2.3.2 that a solution 
initiated by a harmonic function with wavenumber к is 


u(z,t) = f (t)e''* (3.60) 
where f(t) satisfies the ODE 
af = —iakf 
dt 


Solving for f(t) and substituting into Eq. 3.60 gives the exact solution as 
u(x,t) = fOe 


If second-order centered differences are applied to the spatial term, the following 
ODE is obtained for f(t): 


df _ E a 


ба aac ану 


?Note that terms such as (5,u)j;-1 are handled by letting 6,u = ik*e/^/^7" and evaluating the 
shift in j. 
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Solving this ODE exactly (since we are considering the error from the spatial approx- 
imation only) and substituting into Eq. 3.60, we obtain 
(x,t) = Ј(0)е 9 (3.61) 


Unumerical 


where а* is the numerical (or modified) phase speed, which is related to the modified 
wavenumber by 


а kt 
à к 
For the above example, 
а sinkAr 
a KAT 


The numerical phase speed is the speed at which a harmonic function is propagated 
numerically. Since a*/a < 1 for this example, the numerical solution propagates 
too slowly. Since a* is a function of the wavenumber, the numerical approximation 
introduces dispersion, although the original PDE is nondispersive. As a result, a 
waveform consisting of many different wavenumber components eventually loses its 
original form. 

Figure 3.5 shows the numerical phase speed for the schemes considered previously. 
The number of points per wavelength (PPW) by which a given wave is resolved is 
given by 27/кАх. The resolving efficiency of a scheme can be expressed in terms 
of the PPW required to produce errors below a specified level. For example, the 
second-order centered difference scheme requires 80 PPW to produce an error in 
phase speed of less than 0.1 percent. The 5-point fourth-order centered scheme and 
the fourth-order Padé scheme require 15 and 10 PPW respectively to achieve the 
same error level. 

From the form of Eq. 3.61, we see that the real part of the modified wavenumber 
leads to an error in the propagation speed, while the imaginary part leads to an 
error in the amplitude of the solution. Thus the antisymmetric portion of the spatial 
difference operator determines the error in speed and the symmetric portion the error 
in amplitude. This is further discussed in Section 11.2. 


3.6 Difference Operators at Boundaries 


As discussed in Section 3.3.2, a matrix difference operator incorporates both the 
difference approximation in the interior of the domain and that at the boundaries. 
In this section, we consider the boundary operators needed for our model equations 
for convection and diffusion. In the case of periodic boundary conditions, no special 
boundary operators are required. 
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Figure 3.5: Numerical phase speed for various schemes. 


3.6.1 The Linear Convection Equation 


Referring to Section 2.3, the boundary conditions for the linear convection equation 
can be either periodic or of inflow-outflow type. In the latter case, a Dirichlet bound- 
ary condition is given at the inflow boundary, while no condition is specified at the 
outflow boundary. Here we assume that the wave speed a is positive; thus the left- 
hand boundary is the inflow boundary, and the right-hand boundary is the outflow 
boundary. Thus the vector of unknowns is 


T = [и\, us, ..., им] (3.62) 


and ug is specified. 

Consider first the inflow boundary. It is clear that as long as the interior difference 
approximation does not extend beyond u;.;, then no special treatment is required 
for this boundary. For example, with second-order centered differences we obtain 


бй = Aŭ + (bc) (3.63) 


with 


zie ще] 
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However, if we use the fourth-order interior operator given in Eq. 3.54, then the 
approximation at j = 1 requires a value of uj», which is outside the domain. Hence 
a different operator is required at 7 = 1 which extends only to 7 — 1, while having 
the appropriate order of accuracy. Such an operator, known as a numerical boundary 
scheme, can have an order of accuracy which is one order lower than that of the 
interior scheme, and the global accuracy will equal that of the interior scheme. For 
example, with fourth-order centered differences, we can use the following third-order 
operator at j = 1: 


1 
(бәш), = GAZ V uo — Зи + био — из) (3.65) 


which is easily derived using a Taylor table. The resulting difference operator has the 
form of Eq. 3.63 with 





28. 42. 539 — tlg 
—1 —8 0 8 —1 Е —1 ио 
acto) ТУУ Жоо Е (P9) 


This approximation is globally fourth-order accurate. 

At the outflow boundary, no boundary condition is specified. We must approx- 
imate Qu/Ox at node M with no information about им+1. Thus the second-order 
centered-difference operator, which requires u;,,, cannot be used at 7 = M. A back- 
ward difference formula must be used. With a second-order interior operator, the 
following first-order backward formula can be used: 


(беч)м = Azar — uy-1) (3.67) 


This produces a difference operator with 


0 1 _ 
ep 01 2 
=] —1 0 1 “5 E 
A = — bc) = —— 0 3.68 
2A T i ( ©) 2Ах ; 1599) 
—1 0 1 ў 
—2 2 0 


In the case of a fourth-order centered interior operator the last two rows of А require 
modification. 
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Another approach to the development of boundary schemes is in terms of space 
extrapolation. The following formula allows илт to be extrapolated from the interior 
data to arbitrary order on an equispaced grid: 


(1— E-!fuy,,—0 (3.69) 


where E is the shift operator defined by Би; = uj+ı and the order of the approxima- 
tion is p — 1. For example, with p = 2 we obtain 


(1—2E^7 + E’? Jumi = им+1— 2uy + uy = 0 (3.70) 
which gives the following first-order approximation to um+1: 
UM+1 = 2им — UM-1 (3.71) 


Substituting this into the second-order centered difference operator applied at node 
M gives 


1 1 


(Ssu) y = JAg UMH —Um-1) = JAg UM — им-1 — Um-1) 
1 
= Ag M — им-1) (3.72) 


which is identical to Eq. 3.67. 


3.6.2 The Diffusion Equation 


In solving the diffusion equation we must consider Dirichlet and Neumann boundary 
conditions. The treatment of a Dirichlet boundary condition proceeds along the same 
lines as the inflow boundary for the convection equation discussed above. With the 
second-order centered interior operator 


1 
(9.4); = Ag sa — 2и; 435-1) (3.73) 
no modifications are required near boundaries, leading to the difference operator given 


in Eq. 3.27. 
For a Neumann boundary condition, we assume that Ou/Ox is specified at j = 


M + 1, that is 
(22) (=) (3.74) 
Ox hei Ox А 
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Thus we design an operator at node M which is in the following form: 
1 c [ди 
бут = — Won m . 
(и) м = (аим + bum) + z> | Ж (3.75) 


where a, b, and c are constants which can easily be determined using a Taylor table, 
as shown in Table 3.4. 


Table 3.4. Taylor table for Neumann boundary condition. 





Solving for a, b, and c, we obtain the following operator: 


1 2 Qu 
(6-20) у = 5 (2им-1 — 2им) + —— (22) (3.76) 
M 3Ax?2 3Ax \ Oz m 


which produces the difference operator given in Eq. 3.29. Notice that this operator 
is second-order accurate. In the case of a numerical approximation to a Neumann 
boundary condition, this is necessary to obtain a globally second-order accurate for- 
mulation. This contrasts with the numerical boundary schemes described previously 
which can be one order lower than the interior scheme. 

We can also obtain the operator in Eq. 3.76 using the space extrapolation idea. 
Consider a second-order backward-difference approximation applied at node M + 1: 


з 1 
— = ——(uy-i- 4uy + 3um+1) + O(Az?) (3.77) 
(5: nd 2Ar 


Solving for им-+1 gives 


1 д 
"Md = 3 on – им-1 + 2Ах (22) | + O(A2)) (3.78) 
M4 
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Substituting this into the second-order centered difference operator for a second 
derivative applied at node M gives 


1 
1 Qu 
= —— ]3uy_1—6 4 — = 2Ax | — 
3A UM-1 им + UM UM 1c X (=)... 
1 2 Ou 
= —(2Quy-i = 2 — | — 3.80 
злу Е им) + 3Ar E ( ) 


which is identical to Eq. 3.76. 


3.7 Problems 


1. Derive a third-order finite-difference approximation to a first derivative in the 
form 


1 
(буи); = Ag (tui? + buj_1 + cuj + duj+1) 


Find the leading error term. 


2. Derive a finite-difference approximation to a first derivative in the form 


1 
a(ó,u)j 1 + (и); = Az i-i + сиў + duj44) 


Find the leading error term. 


3. Using a 4 (interior) point mesh, write out the 4x4 matrices and the boundary- 
condition vector formed by using the scheme derived in question 2 when both 
u and Qu/Ox are given at j = 0 and u is given at j = 5. 


4. Repeat question 2 with d — 0. 


5. Derive a finite-difference approximation to a third derivative in the form 


(05,0); GUj—2 + buj + сиў + duj41 + CU; +2) 


1 
- As 


Find the leading error term. 
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. Derive a compact (or Padé) finite-difference approximation to a second deriva- 


tive in the form 


1 
d(6,,u);—1 + (бм); + е(ӧ0); = Age uii + bu; + cuj+1) 


Find the leading error term. 


. Find the modified wavenumber for the operator derived in question 1. Plot the 


real and imaginary parts of &* Ax vs. &Az for 0 € &Ax < т. Compare the real 
part with that obtained from the fourth-order centered operator (Eq. 3.54). 


. Application of the second-derivative operator to the function e'^* gives 


e ei^ 
Ox? 


Application of a difference operator for the second derivative gives 


= — Kee 





(056729) = — gift 


thus defining the modified wavenumber к* for a second derivative approxima- 
tion. Find the modified wavenumber for the second-order centered difference 
operator for a second derivative, the noncompact fourth-order operator (Eq. 
3.55), and the compact fourth-order operator derived in question 6. Plot k&*Ax 
vs. KAx for 0 < KAT < m. 


. Find the grid-points-per-wavelength (PPW) requirement to achieve a phase 


speed error less than 0.1 percent for sixth-order noncompact and compact cen- 
tered approximations to a first derivative. 


Consider the following one-sided differencing schemes, which are first-, second-, 
and third-order, respectively: 


(su); = (uj — uj-1) /Ат 
(ди); = (3uj — 4uj-1 + uj-2) /(2Ах) 


(буи); = (11и; — 18и;_1 + 9uj—2 = 2u,;-3) /(6Ax) 


Find the modified wavenumber for each of these schemes. Plot the real and 
imaginary parts of &'Ax vs. &Ax for 0 < &Az < тп. Derive the two leading 
terms in the truncation error for each scheme. 


Chapter 4 


THE SEMI-DISCRETE 
APPROACH 


One strategy for obtaining finite-difference approximations to a PDE is to start by 
differencing the space derivatives only, without approximating the time derivative. 
In the following chapters, we proceed with an analysis making considerable use of 
this concept, which we refer to as the semi-discrete approach. Differencing the space 
derivatives converts the basic PDE into a set of coupled ODE's. In the most general 
notation, these ODE's would be expressed in the form 

= = ЕК(ї,®) (4.1) 
which includes all manner of nonlinear and time-dependent possibilities. On occasion, 
we use this form, but the rest of this chapter is devoted to a more specialized matrix 
notation described below. 

Another strategy for constructing a finite-difference approximation to a PDE is 
to approximate all the partial derivatives at once. This generally leads to a point 
difference operator (see Section 3.3.1) which, in turn, can be used for the time advance 
of the solution at any given point in the mesh. As an example let us consider the 
model equation for diffusion 

ди u 

Ot Ox? 
Using three-point central-differencing schemes for both the time and space derivatives, 
we find 


7 — ued) a E — 2109) + us | 
2h | Aa? 
Or 
n+l n-1 2hv n n n 
up ap. As ub — 2uj + uy” (4.2) 


51 


52 CHAPTER 4. THE SEMI-DISCRETE APPROACH 


Clearly Eq. 4.2 is a difference equation which can be used at the space point j to 
advance the value of u from the previous time levels n and n — 1 to the level n + 1. 
It is a full discretization of the PDE. Note, however, that the spatial and temporal 
discretizations are separable. Thus, this method has an intermediate semi-discrete 
form and can be analyzed by the methods discussed in the next few chapters. 


Another possibility is to replace the value of ut? in the right hand side of Eq. 4.2 


with the time average of u at that point, namely (u^? + Тш This results in 
the formula 


(n+1) , | (n-1) 
бш, eun DE | ш; [ME ru? 


j AT (4.3) 


2 








which can be solved for u”*) and time advanced at the point j. In this case, the 
spatial and temporal discretizations are not separable, and no semi-discrete form 
exists. 


Equation 4.2 is sometimes called Richardson's method of overlapping steps and 
Eq. 4.3 is referred to as the DuFort-Frankel method. As we shall see later on, there 
are subtle points to be made about using these methods to find a numerical solution 
to the diffusion equation. There are a number of issues concerning the accuracy, 
stability, and convergence of Eqs. 4.2 and 4.3 which we cannot comment on until we 
develop a framework for such investigations. We introduce these methods here only to 
distinguish between methods in which the temporal and spatial terms are discretized 
separately and those for which no such separation is possible. For the time being, we 
shall separate the space difference approximations from the time differencing. In this 
approach, we reduce the governing PDE’s to ODE’s by discretizing the spatial terms 
and use the well-developed theory of ODE solutions to aid us in the development of 
an analysis of accuracy and stability. 


4.1 Reduction of PDE’s to ODE’s 


4.1.1 The Model ODE’s 


First let us consider the model PDE's for diffusion and biconvection described in 
Chapter 2. In these simple cases, we can approximate the space derivatives with 
difference operators and express the resulting ODE's with a matrix formulation. This 
is a simple and natural formulation when the ODE's are linear. 
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Model ODE for Diffusion 


For example, using the 3-point central-differencing scheme to represent the second 
derivative in the scalar PDE governing diffusion leads to the following ODE diffusion 
model 


du V > > 
— = —; B(1,—2,1 4.4 
dt aa? И И Ju + (bc) ( ) 


with Dirichlet boundary conditions folded into the (bc) vector. 


Model ODE for Biconvection 


The term biconvection was introduced in Section 2.3. It is used for the scalar con- 
vection model when the boundary conditions are periodic. In this case, the 3-point 
central-differencing approximation produces the ODE model given by 

du а > 

— = —— —B,(—1,0,1)u 4.5 

dt INT »( dus ) ( ) 
where the boundary condition vector is absent because the flow is periodic. 

Eqs. 4.4 and 4.5 are the model ODE's for diffusion and biconvection of a scalar in 

one dimension. They are linear with coefficient matrices which are independent of x 
and t. 


4.1.2 The Generic Matrix Form 


The generic matrix form of a semi-discrete approximation is expressed by the equation 


— — Au — f(t) (4.6) 


Note that the elements in the matrix А depend upon both the PDE and the type of 
differencing scheme chosen for the space terms. The vector ПО, is usually determined 
by the boundary conditions and possibly source terms. In general, even the Euler 
and Navier-Stokes equations can be expressed in the form of Eq. 4.6. In such cases 
the equations are nonlinear, that is, the elements of A depend on the solution 4 and 
are usually derived by finding the Jacobian of a flux vector. Although the equations 
are nonlinear, the linear analysis presented in this book leads to diagnostics that are 
surprisingly accurate when used to evaluate many aspects of numerical methods as 
they apply to the Euler and Navier-Stokes equations. 
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4.2 Exact Solutions of Linear ODE’s 


In order to advance Eq. 4.1 in time, the system of ODE's must be integrated using a 
time-marching method. In order to analyze time-marching methods, we will make use 
of exact solutions of coupled systems of ODE's, which exist under certain conditions. 
The ODE's represented by Eq. 4.1 are said to be linear if F is linearly dependent on 
и (i.e., if OF/Ou = A where A is independent of u). As we have already pointed out, 
when the ODE's are linear they can be expressed in a matrix notation as Eq. 4.6 in 
which the coefficient matrix, A, is independent of и. If А does depend explicitly on 
t, the general solution cannot be written; whereas, if A does not depend explicitly on 
t, the general solution to Eq. 4.6 can be written. This holds regardless of whether or 
not the forcing function, F depends explicitly on t. 

As we shall soon see, the exact solution of Eq. 4.6 can be written in terms of 
the eigenvalues and eigenvectors of A. This will lead us to a representative scalar 
equation for use in analyzing time-marching methods. These ideas are developed in 
the following sections. 


4.2.1 Eigensystems of Semi-Discrete Linear Forms 
Complete Systems 


An M x M matrix is represented by a complete eigensystem if it has a complete set 
of linearly independent eigenvectors (see Appendix A) . An eigenvector, £m, and its 
corresponding eigenvalue, Àm, have the property that 


A, = Аа ог 
[A — Ат» = 0 (4.7) 
The eigenvalues are the roots of the equation 


det[A — AI] = 0 


We form the right-hand eigenvector matrix of a complete system by filling its columns 
with the eigenvectors £m: 
X= m CN Fo ые» I 


The inverse is the left-hand eigenvector matrix, and together they have the property 
that 


X AX = A (4.8) 


where A is a diagonal matrix whose elements are the eigenvalues of A. 


42. EXACT SOLUTIONS OF LINEAR ODE'S 95 


Defective Systems 


If an M x M matrix does not have a complete set of linearly independent eigenvectors, 
it cannot be transformed to a diagonal matrix of scalars, and it is said to be defective. 
It can, however, be transformed to a diagonal set of blocks, some of which may be 
scalars (see Appendix A). In general, there exists some S which transforms any 
matrix А such that 


S AS = Ј 
where 
Л 
Jo 
ems 
Jm 
and 
Àm 1 1 
Je - Ат 
" 1 
MEO 


The matrix J is said to be in Jordan canonical form, and an eigenvalue with multi- 
plicity n within a Jordan block is said to be a defective eigenvalue. Defective systems 
play a role in numerical stability analysis. 


4.2.2 Single ODE’s of First- and Second-Order 
First-Order Equations 


The simplest nonhomogeneous ODE of interest is given by the single, first-order 
equation 


du 


та Au + ae" (4.9) 


where А, a, and и are scalars, all of which can be complex numbers. The equation 
is linear because A does not depend on и, and has a general solution because A does 
not depend on t. It has a steady-state solution if the right-hand side is independent 
of t, i.e., if u = 0, and is homogeneous if the forcing function is zero, i.e., if a = 0. 
Although it is quite simple, the numerical analysis of Eq. 4.9 displays many of the 
fundamental properties and issues involved in the construction and study of most 
popular time-marching methods. This theme will be developed as we proceed. 
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The exact solution of Eq. 4.9 is, for u Æ A, 


ае! 


и — А 





u(t) = ce + 


where c, is a constant determined by the initial conditions. In terms of the initial 
value of u, it can be written 


The interesting question can arise: What happens to the solution of Eq. 4.9 when 
и = A? This is easily found by setting и = А + є, solving, and then taking the limit 
as є + 0. Using this limiting device, we find that the solution to 


d 
— = Au + ае“ (4.10) 


is given by 
u(t) = [u(0) + ое 


As we shall soon see, this solution is required for the analysis of defective systems. 


Second-Order Equations 


The homogeneous form of a second-order equation is given by 


d* d 
T +a + aou 0 (4.11) 


where a; and ао are complex constants. Now we introduce the differential operator 
D such that 


D 


and factor u(t) out of Eq. 4.11, giving 
(D? + a4D + ao) u(t) =0 


The polynomial in D is referred to as a characteristic polynomial and designated 
P(D). Characteristic polynomials are fundamental to the analysis of both ODE’s and 
OAE’s, since the roots of these polynomials determine the solutions of the equations. 
For ODE’s, we often label these roots in order of increasing modulus as Аз, А, ---, Am; 
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---, Ам. They are found by solving the equation P(A) = 0. In our simple example, 
there would be two roots, A, and А, determined from 


P(A) = № нал +а = 0 (4.19) 
and the solution to Eq. 4.11 is given by 
u(t) = ciet + eet (4.13) 


where c, and сә are constants determined from initial conditions. The proof of this 
is simple and is found by substituting Eq. 4.13 into Eq. 4.11. One finds the result 


cet (0 + ал А + ao) + cae? (X2 + аА + ао) 
which is identically zero for all сү, сә, and t if and only if the A's satisfy Eq. 4.12. 


4.2.8 Coupled First-Order ODE's 
A Complete System 


А set of coupled, first-order, homogeneous equations is given by 


ГА 
Чу = аи + ато U2 
ГА 
Чо = аә U1 + a22 U2 (4.14) 
which can be written 
>I > > а а 
T 11 012 
u = Au , uuu , А=(ш)= 
аә Q22 
Consider the possibility that a solution is represented by 
ui = С rye! + Сә пузе? 
uo = с TAE + сә 235€! (4.15) 


By substitution, these are indeed solutions to Eq. 4.14 if and only if 


Q11 @12 її | pe 111 ај] 012 Т12 | _ лә 112 (4.16) 
a21 Q22 T21 T21 a21 Q22 T22 T22 
Notice that a higher-order equation can be reduced to a coupled set of first-order 
equations by introducing a new set of dependent variables. Thus, by setting 
u =u , W =u , 
we find Eq. 4.11 can be written 


uj, = -—dau;-— aguz 
ui (4.17) 


1 
Ug 


which is a subset of Eq. 4.14. 


58 CHAPTER 4. THE SEMI-DISCRETE APPROACH 


A Derogatory System 


Eq. 4.15 is still a solution to Eq. 4.14 if Ay = A» = A, provided two linearly independent 
vectors exist to satisfy Eq. 4.16 with А = А. In this case 


o А0 А æ (0 АА) 


provide such a solution. This is the case where А has a complete set of eigenvectors 
and is not defective. 





A. Defective System 


If A is defective, then it can be represented by the Jordan canonical form 


ul, A 0 ui 
= 4.18 
bends 22 
whose solution is not obvious. However, in this case, one can solve the top equation 


first, giving uı(t) = 1. (0)е^*. Then, substituting this result into the second equation, 
one finds 


ES = Au» + Uy (0)e* 
which is identical in form to Eq. 4.10 and has the solution 
из(®) = [u2(0) + u1 (0)t]e* 
From this the reader should be able to verify that 


u3(t) = (a + bt + c£ e" 


is a solution to 


и ^ ui 
is | eso do А u2 
u3 1 Alius 
if 
1 
a=u3(0) , ф=мш(0) , c= (0) (4.19) 


The general solution to such defective systems is left as an exercise. 
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4.2.4 General Solution of Coupled ODE's with Complete Eigen- 
systems 


Let us consider a set of coupled, nonhomogeneous, linear, first-order ODE’s with 
constant coefficients which might have been derived by space differencing a set of 
PDE's. Represent them by the equation 


> 


a = 44-70) (4.20) 


Our assumption is that the M x M matrix A has a complete eigensystem! and can 


be transformed by the left and right eigenvector matrices, X^! and X, to a diagonal 
matrix A having diagonal elements which are the eigenvalues of A, see Section 4.2.1. 
Now let us multiply Eq. 4.20 from the left by X^! and insert the identity combination 
XX-! = I between А and u. There results 
-1 du =1 -17 -17 

X ТШЕН AX-X u—X f(t) (4.21) 
Since A is independent of both ара t, the elements in X^! and X are also indepen- 
dent of both їй and t, and Eq. 4.21 can be modified to 


OX = AXi- X7 f) 


Finally, by introducing the new variables w and g such that 
w=X u , gt)=X ft) (4.22) 


we reduce Eq. 4.20 to a new algebraic form 


> 


dw > > 

— = Au — g(t 4.23 

m g(t) (4.23) 
It is important at this point to review the results of the previous paragraph. Notice 

that Eqs. 4.20 and 4.23 are expressing exactly the same equality. The only difference 

between them was brought about by algebraic manipulations which regrouped the 

variables. However, this regrouping is crucial for the solution process because Eqs. 


!Tn the following, we exclude defective systems, not because they cannot be analyzed (the example 
at the conclusion of the previous section proves otherwise), but because they are only of limited 
interest in the general development of our theory. 
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4.23 are no longer coupled. They can be written line by line as a set of independent, 
single, first-order equations, thus 


wi = Aw, — gi (t) 
и = A bnc) 


For any given set of gm(t) each of these equations can be solved separately and then 
recoupled, using the inverse of the relations given in Eqs. 4.22: 


u(t) = Xu(t) 


[ 
9 
е 
B 
= 
8 

3 


(4.25) 


where Z,, is the m’th column of X, i.e., the eigenvector corresponding to Am. 

We next focus on the very important subset of Eq. 4.20 when neither A nor f has 
any explicit dependence on #. In such a case, the gm in Eqs. 4.23 and 4.24 are also 
time invariant and the solution to any line in Eq. 4.24 is 


1 
wn (t) = ПВ ке Üm 
Àm 


where the Cm are constants that depend on the initial conditions. Transforming back 
to the u-system gives 


u(t) = Xu(t) 


M 
= Уу) Wal tan 
m=1 
M M 1 К 
= 5 бһе^”* Lm + 5 VC дт т 
т=1 т=1 Xm 
M E E 
= age Em + XAXI 
m=1 
M 
= 5 eue tr. + AIF 
т=1 
(4.26) 
— 





Transient Steady-state 


43. REAL SPACE AND EIGENSPACE 61 


Note that the steady-state solution is A^ f, as might be expected. 

The first group of terms on the right side of this equation is referred to classically 
as the complementary solution or the solution of the homogeneous equations. The 
second group is referred to classically as the particular solution or the particular 
integral. In our application to fluid dynamics, it is more instructive to refer to these 
groups as the transient and steady-state solutions, respectively. An alternative, but 
entirely equivalent, form of the solution is 


u(t) = et ту H-H met Em +- Heut put AIF (4.27) 


4.3 Real Space and Eigenspace 


4.3.1 Definition 


Following the semi-discrete approach discussed in Section 4.1, we reduce the partial 
differential equations to a set of ordinary differential equations represented by the 
generic form 

du > > 

„= Au — f (4.28) 
The dependent variable 4 represents some physical quantity or quantities which relate 
to the problem of interest. For the model problems on which we are focusing most 
of our attention, the elements of A are independent of both u and t. This permits 
us to say a great deal about such problems and serves as the basis for this section. 
In particular, we can develop some very important and fundamental concepts that 
underly the global properties of the numerical solutions to the model problems. How 
these relate to the numerical solutions of more practical problems depends upon the 
problem and, to a much greater extent, on the cleverness of the relator. 

We begin by developing the concept of “spaces”. That is, we identify different 
mathematical reference frames (spaces) and view our solutions from within each. 
In this way, we get different perspectives of the same solution, and this can add 
significantly to our understanding. 

The most natural reference frame is the physical one. We say 


If a solution is expressed in terms of u, it is said 
to be in real space. 


There is, however, another very useful frame. We saw in Sections 4.2.1 and 4.2 that 
pre- and post-multiplication of A by the appropriate similarity matrices transforms A 
into a diagonal matrix, composed, in the most general case, of Jordan blocks or, in the 


62 CHAPTER 4. THE SEMI-DISCRETE APPROACH 


simplest nondefective case, of scalars. Following Section 4.2 and, for simplicity, using 
only complete systems for our examples, we found that Eq. 4.28 had the alternative 
form Е 

20 = Аш – 9 
which is an uncoupled set of first-order ODE's that can be solved independently for 
the dependent variable vector ш. We say 


If a solution is expressed in terms of ш, it is said to 
be in eigenspace (often referred to as wave space). 


The relations that transfer from one space to the other are: 





The elements of u relate directly to the local physics of the problem. However, the 
elements of w are linear combinations of all of the elements of u, and individually 
they have no direct local physical interpretation. 
When the forcing function f is independent of t, the solutions in the two spaces 
are represented by 
u(t) — bE Ау 


апа 


Amt 


1 
Wm(t) = Cme TM m ; m —1,2,-..,M 


т 


for real space and eigenspace, respectively. At this point we make two observations: 


1. the transient portion of the solution in real space consists of a linear combination 
of contributions from each eigenvector, and 


2. the transient portion of the solution in eigenspace provides the weighting of 
each eigenvector component of the transient solution in real space. 


4.3.2 Eigenvalue Spectrums for Model ODE’s 


It is instructive to consider the eigenvalue spectrums of the ODE's formulated by 
central differencing the model equations for diffusion and biconvection. These model 
ODE’s are presented in Section 4.1. Equations for the eigenvalues of the simple 
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tridiagonals, B(M : a,b,c) and B,(M : a,b, c), are given in Appendix B. From Section 
B.1, we find for the model diffusion equation with Dirichlet boundary conditions 





и MT 
= —,/-24+2 
t) 
—Av , 5 тт 
= ——-si ИН ; =1,2,---,M 4.29 
Ar? sin (zar) 7 т К Мн; 3 ( ) 
and, from Section B.4, for the model biconvection equation 
=% 2 
| EE A sin (==) Mec Me 
= —їк a m=0,1,---,;M-1 (4.30) 
where 
Д 
LEE mnm so; M eT (4.31) 


is the modified wavenumber from Section 3.5, km = m, and Ax = 27/ М. Notice 
that the diffusion eigenvalues are real and negative while those representing periodic 
convection are all pure imaginary. The interpretation of this result plays a very 
important role later in our stability analysis. 


4.3.8 Eigenvectors of the Model Equations 


Next we consider the eigenvectors of the two model equations. These follow as special 
cases from the results given in Appendix B. 


'The Diffusion Model 


Consider Eq. 4.4, the model ODE's for diffusion. First, to help visualize the matrix 
structure, we present results for a simple 4-point mesh and then we give the general 
case. The right-hand eigenvector matrix X is given by 


sin (xı) sin(2zi) ѕір (321) sin (421) 

sin (rj) sin (2x2) sin (8x2) sin (4x3) 

sin (r3) sin(2z3) sin (8x3) sin (4x3) 

sin (x4) sin (224) sin (324) sin (4x4) 
The columns of the matrix are proportional to the eigenvectors. Recall that 2; = 
jAx = jn/(M +1), so in general the relation @ = X сап be written as 


M 
из = M wmsinmz; ; j=1,2,---,M (4.32) 


m=1 
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For the inverse, or left-hand eigenvector matrix X^!, we find 


sin (zi) sin (z2) sin (23) sin (24) 

sin (2x1) sin (2х5) sin (2х3) sin (224) 

sin (321) sin (3x2) sin (3x3) sin (324) 

sin (41у) sin (419) sin (4x3) sin (4x4) 
The rows of the matrix are proportional to the eigenvectors. In general 10 = X-!d 
gives 


Wm = Уи; sin ma; >; m=1,2,---,M (4.33) 


In the field of harmonic analysis, Eq. 4.33 represents a sine transform of the func- 
tion u(x) for an M-point sample between the boundaries z = 0 and x = m with the 
condition u(0) = u(x) = 0. Similarly, Eq. 4.32 represents the sine synthesis that 
companions the sine transform given by Eq. 4.33. In summary, 


For the model diffusion equation: 


is a sine transform from real space to (sine) wave 


space. 


is a sine synthesis from wave space back to real 
space. 





The Biconvection Model 


Next consider the model ODE's for periodic convection, Eq. 4.5. Тһе coefficient 
matrices for these ODE's are always circulant. For our model ODE, the right-hand 
eigenvectors are given by 


ЕН = gii Qnm/M) j = 0,1, eM 
Yo 50, d oes M SI 
With т; 2j: Ax = j- 2n/M, we can write à = Xi as 
М-1 | 
uj = » 0те") , 7 E 0,1,- "М —1 (4.34) 


m=0 


4.3. REAL SPACE AND EIGENSPACE 65 


For a 4-point periodic mesh, we find the following left-hand eigenvector matrix 
from Appendix B.4: 


un 1 1 1 1 uUi 
шә 1 |1 e72iT/4 e74iT/4 e 6in/A4 Us sacs 
из| 4|1 e 5^ eg-8m4 gp 12i7/4 | | yg | = X u 
Wa 1 en Sin/4 е-121т/4 е-18т/4 | | q, 


In general 
1 M-1 | 
Um = aT у oue m ; m=0,1,---,M-—1 
j—0 
This equation is identical to a discrete Fourier transform of the periodic dependent 
variable u using an M-point sample between and including x = 0 and x = 2r — Az. 


For circulant matrices, it is straightforward to establish the fact that the relation 
u= Xù represents the Fourier synthesis of the variable w back to u. In summary, 


For any circulant system: 


is a complex Fourier transform from real space to 
wave space. 


is a complex Fourier synthesis from wave space 
back to real space. 





4.3.4 Solutions of the Model ОРЕ?” 


We can now combine the results of the previous sections to write the solutions of our 
model ODE's. 


The Diffusion Equation 


For the diffusion equation, Eq. 4.27 becomes 
M 
= M ese! sin mz; + (A f);, 4=1,2,---,M (4.35) 


where 


—4v . 5 MT 
ы ез ро 4. 
ac ын (диг) (00) 
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With the modified wavenumber defined as 


2 A 
Ky, = = sin (25 =) (4.37) 





and using Km = m, we can write the ODE solution as 
M * 2 
и;@ = У eue" *sinez + (A7! f), qe 12M (4.38) 
m=1 


This can be compared with the exact solution to the PDE, Eq. 2.37, evaluated at the 
nodes of the grid: 


M 
u;(t) = У; спек" t sin Kmi + h(z;), j=1,2,---, M (4.39) 


m=1 


We see that the solutions are identical except for the steady solution and the 
modified wavenumber in the transient term. The modified wavenumber is an approx- 
imation to the actual wavenumber. The difference between the modified wavenumber 
and the actual wavenumber depends on the differencing scheme and the grid resolu- 
tion. This difference causes the various modes (or eigenvector components) to decay 
at rates which differ from the exact solution. With conventional differencing schemes, 
low wavenumber modes are accurately represented, while high wavenumber modes (if 
they have significant amplitudes) can have large errors. 


The Convection Equation 
For the biconvection equation, we obtain 
M-1 | 
ult) = >, opem j—0,1,---, M-1 (4.40) 


m=0 
where 
Am, = cC (4.41) 
with the modified wavenumber defined in Eq. 4.31. We can write this ODE solution 
as 
М-1 


u,(t) = сле T mtemi j=0,1,---,M-1 4.42 
2 


m=0 
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and compare it to the exact solution of the PDE, Eq. 2.26, evaluated at the nodes of 
the grid: 


M-1 
ujt) = Y fale Certs: 30i Ma (4.43) 
m=0 


Once again the difference appears through the modified wavenumber contained in 
Ам. As discussed in Section 3.5, this leads to an error in the speed with which various 
modes are convected, since к* is real. Since the error in the phase speed depends on 
the wavenumber, while the actual phase speed is independent of the wavenumber, 
the result is erroneous numerical dispersion. In the case of non-centered differencing, 
discussed in Chapter 11, the modified wavenumber is complex. The form of Eq. 4.42 
shows that the imaginary portion of the modified wavenumber produces nonphysical 
decay or growth in the numerical solution. 


4.4 The Representative Equation 
In Section 4.3, we pointed out that Eqs. 4.20 and 4.23 express identical results but in 


terms of different groupings of the dependent variables, which are related by algebraic 
manipulation. This leads to the following important concept: 


The numerical solution to a set of linear ODE's (in which A is 
not a function of t) is entirely equivalent to the solution obtained 


if the equations are transformed to eigenspace, solved there in 
their uncoupled form, and then returned as a coupled set to real 
space. 





The importance of this concept resides in its message that we can analyze time- 
marching methods by applying them to a single, uncoupled equation and our con- 
clusions will apply in general. This is helpful both in analyzing the accuracy of 
time-marching methods and in studying their stability, topics which are covered in 
Chapters 6 and 7. 

Our next objective is to find a "typical" single ODE to analyze. We found the 
uncoupled solution to a set of ODE's in Section 4.2. A typical member of the family 
is 


dWm 


dp = №0 — Gm(t) (4.44) 


The goal in our analysis is to study typical behavior of general situations, not partic- 
ular problems. For such a purpose Eq. 4.44 is not quite satisfactory. The role of Am is 
clear; it stands for some representative eigenvalue in the original A matrix. However, 
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the question is: What should we use for g,,(t) when the time dependence cannot be 
ignored? To answer this question, we note that, in principle, one can express any one 
of the forcing terms g,,(t) as a finite Fourier series. For example 


—g(t) zx У аке? 
k 


for which Eq. 4.44 has the exact solution: 

аке“ 

w(t) = се + a 
(0) pik A 

From this we can extract the k’th term and replace ik with u. This leads to 


The Representative ODE 


А (4.45) 


а 


= Aw + ae" 





which can be used to evaluate all manner of time-marching methods. In such evalua- 
tions the parameters А and u must be allowed to take the worst possible combination 
of values that might occur in the ODE eigensystem. The exact solution of the repre- 
sentative ODE is (for и Z A): 


(4.46) 


4.5 Problems 


1. Consider the finite-difference operator derived in question 1 of Chapter 3. Using 
this operator to approximate the spatial derivative in the linear convection equa- 
tion, write the semi-discrete form obtained with periodic boundary conditions 
on a 5-point grid (M — 5). 


2. Consider the application of the operator given in Eq. 3.52 to the 1-D diffusion 
equation with Dirichlet boundary conditions. Write the resulting semi-discrete 
ODE form. Find the entries in the boundary-condition vector. 


3. Write the semi-discrete form resulting from the application of second-order cen- 
tered differences to the following equation on the domain 0 € x < 1 with 
boundary conditions u(0) = 0, u(1) = 1: 
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4. Consider a grid with 10 interior points spanning the domain 0 € x < л. For 
initial conditions u(r, 0) = sin(mz) and boundary conditions u(0,t) = u(r, t) = 
0, plot the exact solution of the diffusion equation with v — 1 at t — 1 with 
m = 1 and m = 3. (Plot the solution at the grid nodes only.) Calculate the 
corresponding modified wavenumbers for the second-order centered operator 
from Eq. 4.37. Calculate and plot the corresponding ODE solutions. 


5. Consider the matrix 
А=—БВ,(10,—1,0,1)/(2Ах) 


corresponding to the ODE form of the biconvection equation resulting from the 
application of second-order central differencing on a 10-point grid. Note that 
the domain is 0 € z < 2x and Ах = 27/10. The grid nodes are given by 
ж) = ЈАт, j = 0,1,...9. The eigenvalues of the above matrix A, as well as the 
matrices X and X-!, can be found from Appendix B.4. Using these, compute 
and plot the ODE solution at t = 27 for the initial condition u(r, 0) = sin z. 
Compare with the exact solution of the PDE. Calculate the numerical phase 
speed from the modified wavenumber corresponding to this initial condition 
and show that it is consistent with the ODE solution. Repeat for the initial 
condition u(r,0) = sin 22. 
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Chapter 5 


FINITE-VOLUME METHODS 


In Chapter 3, we saw how to derive finite-difference approximations to arbitrary 
derivatives. In Chapter 4, we saw that the application of a finite-difference approxi- 
mation to the spatial derivatives in our model PDE’s produces a coupled set of ODE's. 
In this Chapter, we will show how similar semi-discrete forms can be derived using 
finite-volume approximations in space. Finite-volume methods have become popu- 
lar in CFD as a result, primarily, of two advantages. First, they ensure that the 
discretization is conservative, i.e., mass, momentum, and energy are conserved in a 
discrete sense. While this property can usually be obtained using a finite-difference 
formulation, it is obtained naturally from a finite-volume formulation. Second, finite- 
volume methods do not require a coordinate transformation in order to be applied on 
irregular meshes. As a result, they can be applied on unstructured meshes consisting 
of arbitrary polyhedra in three dimensions or arbitrary polygons in two dimensions. 
This increased flexibility can be used to great advantage in generating grids about 
arbitrary geometries. 


Finite-volume methods are applied to the integral form of the governing equations, 
either in the form of Eq. 2.1 or Eq. 2.2. Consistent with our emphasis on semi-discrete 
methods, we will study the latter form, which is 


d 
= dV $ FdS = / PdV 51 
dt [m UV sq V(t) dy 


We will begin by presenting the basic concepts which apply to finite-volume strategies. 
Next we will give our model equations in the form of Eq. 5.1. This will be followed 
by several examples which hopefully make these concepts clear. 
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5.1 Basic Concepts 


The basic idea of a finite-volume method is to satisfy the integral form of the con- 
servation law to some degree of approximation for each of many contiguous control 
volumes which cover the domain of interest. Thus the volume V in Eq. 5.1 is that of a 
control volume whose shape is dependent on the nature of the grid. In our examples, 
we will consider only control volumes which do not vary with time. Examining Eq. 
5.1, we see that several approximations must be made. The flux is required at the 
boundary of the control volume, which is a closed surface in three dimensions and a 
closed contour in two dimensions. This flux must then be integrated to find the net 
flux through the boundary. Similarly, the source term P must be integrated over the 
control volume. Next a time-marching method! can be applied to find the value of 


a QdV (5.2) 


at the next time step. 
Let us consider these approximations in more detail. First, we note that the 
average value of Q in a cell with volume V is 


"ES 
Qz | Qav (5.3) 


and Eq. 5.1 can be written as 


d - 
VQ ESTE = [ Pav (5.4) 


for a control volume which does not vary with time. Thus after applying a time- 
marching method, we have updated values of the cell-averaged quantities Q. In order 
to evaluate the fluxes, which are a function of Q, at the control-volume boundary, Q 
can be represented within the cell by some piecewise approximation which produces 
the correct value of Q. This is a form of interpolation often referred to as recon- 
struction. As we shall see in our examples, each cell will have a different piecewise 
approximation to Q. When these are used to calculate F(Q), they will generally 
produce different approximations to the flux at the boundary between two control 
volumes, that is, the flux will be discontinuous. A nondissipative scheme analogous 
to centered differencing is obtained by taking the average of these two fluxes. Another 
approach known as flux-difference splitting is described in Chapter 11. 
The basic elements of a finite-volume method are thus the following: 


l'Time-marching methods will be discussed in the next chapter. 
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1. Given the value of Q for each control volume, construct an approximation to 
Q(r,y,z) in each control volume. Using this approximation, find Q at the 
control-volume boundary. Evaluate F(Q) at the boundary. Since there is a 
distinct approximation to Q(z, y, 2) in each control volume, two distinct values 
of the flux will generally be obtained at any point on the boundary between two 
control volumes. 


2. Apply some strategy for resolving the discontinuity in the flux at the control- 
volume boundary to produce a single value of F(Q) at any point on the bound- 
ary. This issue is discussed in Section 11.4.2. 


3. Integrate the flux to find the net flux through the control-volume boundary 
using some sort of quadrature. 


4. Advance the solution in time to obtain new values of Q. 


The order of accuracy of the method is dependent on each of the approximations. 
These ideas should be clarified by the examples in the remainder of this chapter. 

In order to include diffusive fluxes, the following relation between VQ and Q is 
sometimes used: 


/ VQdv = $ nQds (5.5) 
V S 
Or, in two dimensions, 


f VQdA = $, nQdl (5.6) 


where the unit vector n points outward from the surface or contour. 


5.2 Model Equations in Integral Form 


5.2.1 The Linear Convection Equation 


A two-dimensional form of the linear convection equation can be written as 


Ou Ou . „ди 
ag ae (5.7) 


This PDE governs a simple plane wave convecting the scalar quantity, u(x, y, t) with 
speed a along a straight line making an angle 0 with respect to the x-axis. The 
one-dimensional form is recovered with 0 — 0. 
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For unit speed a, the two-dimensional linear convection equation is obtained from 
the general divergence form, Eq. 2.3, with 


(Q- = и (5.8) 
Е = iucosé + jusin0 (5. 
Р = 0 (5.10) 


Since Q is a scalar, F is simply а vector. Substituting these expressions into a two- 
dimensional form of Eq. 2.2 gives the following integral form 


d . TN u 
"^ /, udA + f. n.(iu cos0 + jusin0)ds = 0 (5.11) 
where A is the area of the cell which is bounded by the closed contour C. 


5.2.2 The Diffusion Equation 


The integral form of the two-dimensional diffusion equation with no source term and 
unit diffusion coefficient v is obtained from the general divergence form, Eq. 2.3, with 


Q = u 
F = -Vu 


= RAEE 
Ox Jay 


Р = 0 


12 
13 


5. 


(5.12) 
(5.13) 
(5.14) 
(5.15) 


Using these, we find 


d .Qu „ðu 


to be the integral form of the two-dimensional diffusion equation. 


5.3 One-Dimensional Examples 


We restrict our attention to a scalar dependent variable и and a scalar flux f, as in 
the model equations. We consider an equispaced grid with spacing Az. The nodes of 
the grid are located at x; = Ј Ат as usual. Control volume j extends from ж; — Ах /2 
to x; + Ax/2, as shown in Fig. 5.1. We will use the following notation: 


= Zj SA ag) = Ах/2 (5.17) 
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1-12 j+1/2 
Ах. 
e e | © | © © 
2 ji ў jd j42 
L R L R 


Figure 5.1: Control volume in one dimension. 














Uj+1/2 = u(1j41/2), fj41/2 = f (uj-41/2) (5.18) 
With these definitions, the cell-average value becomes 
_ 1 Lj41/2 
u(t) = << n u(a, t)dx (5.19) 
and the integral form becomes 
d 23+1/2 
(229) + /)+1/2— fi-1/2 = f l , Рах (5.20) 
vj—1/2 


Now with £ = x — ту, we can expand u(x) in Eq. 5.19 in a Taylor series about т; 
(with t fixed) to get 


_ 0 0]. үм? | ди E (3u & (Pu | 
üj = ce B eH) +5 К аз) +. d£ 





Az? f *u Azt (tu 6 
pg ($5), + 1920 (би) «0 ) (5.21) 
Or 
uj; = uj + O(Az’) (5.22) 


where и; is the value at the center of the cell. Hence the cell-average value and the 
value at the center of the cell differ by a term of second order. 


5.3.1 A Second-Order Approximation to the Convection Equa- 
tion 


In one dimension, the integral form of the linear convection equation, Eq. 5.11, be- 
comes 


dit; 
Ars + Fia LI fi-1/2 =0 (5.23) 
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with f = и. We choose a piecewise constant approximation to u(x) in each cell such 
that 


и(2) = üj 2-1 Š X < 012 (5.24) 
Evaluating this at j + 1/2 gives 
fia = Ригу) = ү; = uj (5.25) 


where the L indicates that this approximation to fj+1/2 is obtained from the approx- 
imation to u(x) in the cell to the left of £j41/2, as shown in Fig. 5.1. The cell to the 
right of £j+1/2, which is cell 7 + 1, gives 


fan = uj (5.26) 
Similarly, cell j is the cell to the right of x;_1/2, giving 
fia =t (5.27) 
and cell j — 1 is the cell to the left of z; 4/5», giving 
Fi wa = Uj-1 (5.28) 


We have now accomplished the first step from the list in Section 5.1; we have defined 
the fluxes at the cell boundaries in terms of the cell-average data. In this example, 
the discontinuity in the flux at the cell boundary is resolved by taking the average of 
the fluxes on either side of the boundary. Thus 


^ 1 l,. 2 

fixa = 3 e + fao) = 508) + üjq1) (5.29) 
апа 

(4 1 L R bs = 

Fiz = g (ia + fia) = 50-1 + uj) (5.30) 


where f denotes a numerical flux which is an approximation to the exact flux. 
Substituting Eqs. 5.29 and 5.30 into the integral form, Eq. 5.23, we obtain 


© l Ч: 
Aq — + (0; + üj41) == z(üji + uj) = Ат + gn = ues) == 0 (5.31) 
With periodic boundary conditions, this point operator produces the following semi- 
discrete form: 

di 1 


4 = 5428-0104 (5.32) 
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This is identical to the expression obtained using second-order centered differences, 
except it is written in terms of the cell average й, rather than the nodal values, g. 
Hence our analysis and understanding of the eigensystem of the matrix B,(—1, 0, 1) 
is relevant to finite-volume methods as well as finite-difference methods. Since the 
eigenvalues of B,(—1,0,1) are pure imaginary, we can conclude that the use of the 
average of the fluxes on either side of the cell boundary, as in Eqs. 5.29 and 5.30, can 
lead to a nondissipative finite-volume method. 


5.3.2 A Fourth-Order Approximation to the Convection Equa- 
tion 


Let us replace the piecewise constant approximation in Section 5.3.1 with a piecewise 
quadratic approximation as follows 


u(£) = a£? + be +e (5.33) 


where € is again equal to x — ту. The three parameters a, b, and c are chosen to 
satisfy the following constraints: 


—Aa/2 
: / u(£)d& = 03-1 


Ax /—зАх/2 

1 Az/2 
use. = п; ‚44 
AT GE: “j (i) 


1 3Aa/2 _ 
Ral се 


These constraints lead to 
_ üja 20) + üj- 
2Аа? 


= Шў+1 — Uj-1 


D (5.35) 





Uj-1 | 26u; Uj+i 
24 


With these values of a, b, and c, the piecewise quadratic approximation produces 
the following values at the cell boundaries: 


C= 


m ANS 
Us ia = guia + 5u; — us) (5.36) 
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1 
ut тро = go n + 5ü; + 2ü; 1) (5.37) 
R Ix » И 
иар = gts + üj + 20) (5.38) 
г M cs Ы - 
Uj-iyo = gu + 5031 — Uj-2) (5.39) 


using the notation defined in Section 5.3.1. Recalling that f = и, we again use the 
average of the fluxes on either side of the boundary to obtain 


^ 1 

fias = 3 V (uiuo) Hs Fu ы] 
1, _ А "NE 

= Tp (tite + Ttij41 + Tu; — Uj-1) (5.40) 


and 


^ 


1 

fi-1/2 = 3 V (uiuo) p f (uk 5) 
1 

= ig a + Tus + TUj-1 = U;-2) (5.41) 





Substituting these expressions into the integral form, Eq. 5.23, gives 
З 1 т, =. Д p 
Ax —— + —(—ti;42 + 8041 = 8Uj—ı + 0-9) = 0 (5.42) 


This is a fourth-order approximation to the integral form of the equation, as can be 
verified using Taylor series expansions (see question 1 at the end of this chapter). 
With periodic boundary conditions, the following semi-discrete form is obtained: 


di 1 = 
— = ————В,(1, – —1)u 5.43 
dt 12Ar »( , 8, 0, 8, )u ( ) 


'This is a system of ODE's governing the evolution of the cell-average data. 


5.3.3 A Second-Order Approximation to the Diffusion Equa- 
tion 
In this section, we describe two approaches to deriving a finite-volume approximation 


to the diffusion equation. The first approach is simpler to extend to multidimensions, 
while the second approach is more suited to extension to higher order accuracy. 
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In one dimension, the integral form of the diffusion equation, Eq. 5.16, becomes 


dü; 
Ar- =F Fia БЕ fj-1/2 = 0 (5.44) 
with f = —Vu = —ди/дх. Also, Eq. 5.6 becomes 
—dzx = u(b) — u(a) (5.45) 
We can thus write the following expression for the average value of the gradient of u 
over the interval rj € £ < 231: 


1 ftin Ou 1 

— da = ——(Uj+1 — Uj 5.46 
А5, dat = pn) (5-46) 
From Eq. 5.22, we know that the value of a continuous function at the center of a given 
interval is equal to the average value of the function over the interval to second-order 
accuracy. Hence, to second-order, we can write 


А ди 1 
faa =- (=) = ——— (üj,,— üj) (5.47) 
2 Ox jua Ах“? i 
Similarly, 
A ji 
fj-1/2 = ET — Чу—1) (5.48) 


Substituting these into the integral form, Eq. 5.44, we obtain 


dü; PN MS hi Sa 
ar = Ay j-1 = 2u; + Uj41) (5.49) 


or, with Dirichlet boundary conditions, 


A 


du 1 "m 
4 = Ae BU -2, E+ (bc) (5.50) 
'This provides a semi-discrete finite-volume approximation to the diffusion equation, 
and we see that the properties of the matrix B(1, —2, 1) are relevant to the study of 
finite-volume methods as well as finite-difference methods. 
For our second approach, we use a piecewise quadratic approximation as in Section 
5.3.2. From Eq. 5.33 we have 


ðu ди 


Xon (5.51) 
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with a and b given in Eq. 5.35. With f = —Ou/Oz, this gives 


Ж Fe а 
fun = Ium = —ÁAg Üi — йу) (5.52) 
R L la 
Jj iy = Jj 1уә = —Ags =) (5.53) 
Notice that there is по discontinuity in the flux at the cell boundary. This produces 
dü; 1, "P 
т, — Az (0-1 — 2uj + Uj41) (5.54) 


which is identical to Eq. 5.49. The resulting semi-discrete form with periodic bound- 
ary conditions is 
du _ 1 
dt Az? 


which is written entirely in terms of cell-average data. 


By(1, —2,1)ü (5.55) 


5.4 A Two-Dimensional Example 


The above one-dimensional examples of finite-volume approximations obscure some 
of the practical aspects of such methods. Thus our final example is a finite-volume 
approximation to the two-dimensional linear convection equation on a grid consisting 
of regular triangles, as shown in Figure 5.2. As in Section 5.3.1, we use a piecewise 
constant approximation in each control volume and the flux at the control volume 
boundary is the average of the fluxes obtained on either side of the boundary. The 
nodal data are stored at the vertices of the triangles formed by the grid. The control 
volumes are regular hexagons with area A, A is the length of the sides of the triangles, 
and £ is the length of the sides of the hexagons. The following relations hold between 
£, A, and A. 


es 
ll 

| 
[> 


mls x 
ко 


= 3 (5.56) 


The two-dimensional form of the conservation law is 


d 
£ | QdA+ ў n Fal -4 (5.57) 
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Figure 5.2: Triangular grid. 


where we have ignored the source term. The contour in the line integral is composed 
of the sides of the hexagon. Since these sides are all straight, the unit normals can 
be taken outside the integral and the flux balance is given by 


d 5 
c dA „|ва = 
К В ] 1= 0 


where v indexes a side of the hexagon, as shown in Figure 5.2. A list of the normals 
for the mesh orientation shown is given in Table 5.1. 


Side,» Outward Normal, n 
(i— м3 j)/2 


(i+ v3 j)/2 
(~i + v3 j)/2 


стл о мн © 


Table 5.1. Outward normals, see Fig. 5.2. 
i and j are unit normals along x and y, respectively. 
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For Eq. 5.11, the two-dimensional linear convection equation, we have for side v 


£/2 


n, - / ға = n, · (1с050 + jsin 0) Е и,(&)а& (5.58) 


where € is a length measured from the middle of a side v. Making the change of 
variable 2 = €/€, one has the expression 


0/2 1/2 
/ u(£)dt = £ f u(z)dz (5.59) 
~£/2 ~1/2 
Then, in terms of u and the hexagon area A, we have 
d 2 1/2 
ED ie udA+ 228) - (1с050 + jsin 0) ef vie , = 0 (5.60) 


The values of n, · (1cos0 + jsin 0) are given by the expressions in Table 5.2. There 
are no numerical approximations in Eq. 5.60. That is, if the integrals in the equation 
are evaluated exactly, the integrated time rate of change of the integral of u over the 
area of the hexagon is known exactly. 


Side,»  n,-(icos0 + jsin0) 


(cos0 — уЗ віп 0)/2 
cos 0 
(соз Ө + V/3sin 0) /2 
(— cos 0 + УЗ і 0)/2 


— cos 0 


(— cos 0 — v3 sin 0) /2 





Table 5.2. Weights of flux integrals, see Eq. 5.60. 


Introducing the cell average, 
/ u dA = Ай, (5.61) 
А 


and the piecewise-constant approximation и = й over the entire hexagon, the ap- 
proximation to the flux integral becomes trivial. Taking the average of the flux on 
either side of each edge of the hexagon gives for edge 1: 


] «e = “pe f да аза (5.62) 
1 2 1/2 2 
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Similarly, we have for the other five edges: 


/ u(z)dz = Bo (5.63) 
[uaz = = (5.64) 
[ oi = Lá (5.65) 
[uae = TRI (5.66) 


Up + Ur 


/ u(z)dz = “BoE (5.67) 


ATE 4 AQ dis ar esaet (oos Be ED Gy s) 
+(—cos@ + V3sin0)(u, — u5)] = 0 (5.68) 
or 
dUy 


1 ME t fox 
Pu зд? cos 0) (tia — tia) + (cos0 + //3вїп 0)(ü, — te) 
+(— cos 0 + V/3sin0)(u, — üj)] = 0 (5.69) 


The reader can verify, using Taylor series expansions, that this is a second-order 
approximation to the integral form of the two-dimensional linear convection equation. 


5.5 Problems 


1. Use Taylor series to verify that Eq. 5.42 is a fourth-order approximation to Eq. 
5.23. 


2. Find the semi-discrete ODE form governing the cell-average data resulting from 
the use of a linear approximation in developing a finite-volume method for the 
linear convection equation. Use the following linear approximation: 


u(£) = a£ +b 
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where b — u; and 


Ш — ja 
a T ——————————— 
2A^- 


and use the average flux at the cell interface. 


3. Using the first approach given in Section 5.3.3, derive a finite-volume approx- 
imation to the spatial terms in the two-dimensional diffusion equation on a 
square grid. 


4. Repeat question 3 for a grid consisting of equilateral triangles. 


Chapter 6 


TIME-MARCHING METHODS 
FOR ODE’S 


After discretizing the spatial derivatives in the governing PDE's (such as the Navier- 
Stokes equations), we obtain a coupled system of nonlinear ODE's in the form 
Ед exu X) (6.1) 
These can be integrated in time using a time-marching method to obtain a time- 
accurate solution to an unsteady flow problem. For a steady flow problem, spatial 
discretization leads to a coupled system of nonlinear algebraic equations in the form 
F(a) = 0 (6.2) 

As a result of the nonlinearity of these equations, some sort of iterative method is 
required to obtain a solution. For example, one can consider the use of Newton's 
method, which is widely used for nonlinear algebraic equations (See Section 6.10.3.). 
This produces an iterative method in which a coupled system of linear algebraic 
equations must be solved at each iteration. These can be solved iteratively using 
relaxation methods, which will be discussed in Chapter 9, or directly using Gaussian 
elimination or some variation thereof. 

Alternatively, one can consider a time-dependent path to the steady state and use 
a time-marching method to integrate the unsteady form of the equations until the 
solution is sufficiently close to the steady solution. The subject of the present chapter, 
time-marching methods for ODE's, is thus relevant to both steady and unsteady flow 
problems. When using a time-marching method to compute steady flows, the goal is 
simply to remove the transient portion of the solution as quickly as possible; time- 
accuracy is not required. This motivates the study of stability and stiffness, topics 
which are covered in the next two chapters. 
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Application of a spatial discretization to a PDE produces a coupled system of 
ODE’s. Application of a time-marching method to an ODE produces an ordinary 
difference equation (OAE ). In earlier chapters, we developed exact solutions to our 
model PDE's and ODE's. In this chapter we will present some basic theory of linear 
OAE’s which closely parallels that for linear ODE's, and, using this theory, we will 
develop exact solutions for the model OAE's arising from the application of time- 
marching methods to the model ODE's. 


6.1 Notation 


Using the semi-discrete approach, we reduce our PDE to a set of coupled ODE’s 
represented in general by Eq. 4.1. However, for the purpose of this chapter, we need 
only consider the scalar case 


— =u = Fut) (6.3) 


Although we use u to represent the dependent variable, rather than w, the reader 
should recall the arguments made in Chapter 4 to justify the study of a scalar ODE. 
Our first task is to find numerical approximations that can be used to carry out the 
time integration of Eq. 6.3 to some given accuracy, where accuracy can be measured 
either in a local or a global sense. We then face a further task concerning the numerical 
stability of the resulting methods, but we postpone such considerations to the next 
chapter. 

In Chapter 2, we introduced the convention that the n subscript, or the (n) su- 
perscript, always points to a discrete time value, and h represents the time interval 
At. Combining this notation with Eq. 6.3 gives 


u, = Fa = F(un,tn) ; tn=nh 


Often we need a more sophisticated notation for intermediate time steps involving 
temporary calculations denoted by à, й, etc. For these we use the notation 


Ша = nta — (аа ty + ah) 
The choice of u’ or F to express the derivative in a scheme is arbitrary. They are 
both commonly used in the literature on ODE's. 

The methods we study are to be applied to linear or nonlinear ODE's, but the 
methods themselves are formed by linear combinations of the dependent variable and 
its derivative at various time intervals. They are represented conceptually by 


Un+1 = HOTAR буш, (METER 111, 09Un, Q—1n-1,77 ) (6.4) 
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With an appropriate choice of the o/s and fs, these methods can be constructed 
to give a local Taylor series accuracy of any order. The methods are said to be 
explicit if 04 = 0 and implicit otherwise. An explicit method is one in which the new 
predicted solution is only a function of known data, for example, ши, и” 4, Un, and 
Un—ı for a method using two previous time levels, and therefore the time advance is 
simple. For an implicit method, the new predicted solution is also a function of the 
time derivative at the new time level, that is, шуу. As we shall see, for systems of 
ODE’s and nonlinear problems, implicit methods require more complicated strategies 
to solve for un+ı than explicit methods. 


6.2 Converting Time-Marching Methods to OAE’s 


Examples of some very common forms of methods used for time-marching general 
ODE's are: 





Until = Un + hw, (6.5) 
Ип+1 = Un + Би (6.6) 
and 
йлы = Unt hu, 
1 
Un+1 = glun + Un+1 + hi, i] (6.7) 


According to the conditions presented under Eq. 6.4, the first and third of these are 
examples of explicit methods. We refer to them as the explicit Euler method and the 
MacCormack predictor-corrector method,! respectively. The second is implicit and 
referred to as the implicit (or backward) Euler method. 

These methods are simple recipes for the time advance of a function in terms of its 
value and the value of its derivative, at given time intervals. The material presented 
in Chapter 4 develops a basis for evaluating such methods by introducing the concept 


of the representative equation 
du А 
— = wu! = Аи + aet 6.8 
7 (6.8) 


written here in terms of the dependent variable, u. The value of this equation arises 
from the fact that, by applying a time-marching method, we can analytically convert 


'Here we give only MacCormack’s time-marching method. The method commonly referred to as 
MacCormack's method, which is a fully-discrete method, will be presented in Section 11.3 
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such a linear ODE into a linear OAE . The latter are subject to a whole body of 
analysis that is similar in many respects to, and just as powerful as, the theory of 
ODE’s. We next consider examples of this conversion process and then go into the 
general theory on solving OAE's. 
Apply the simple explicit Euler scheme, Eq. 6.5, to Eq. 6.8. There results 
Ungi = Un + h(Aun + ael?) 
ог 


ungi — (1 + Ah)us = hae? (6.9) 


Eq. 6.9 is a linear ОЛЕ , with constant coefficients, expressed in terms of the depen- 
dent variable un and the independent variable n. As another example, applying the 
implicit Euler method, Eq. 6.6, to Eq. 6.8, we find 


Until = Un + ҺА + дени) 
Or 
(1— Ah)us 44 — Un = Һе" - ge^" (6.10) 
As a final example, the predictor-corrector sequence, Eq. 6.7, gives 
lina —(1+Ah)u, = ahe” 


1 1 1 
-350 + Аһ)й + Uny — уш = һе (6.11) 


which is a coupled set of linear OAE's with constant coefficients. Note that the first 
line in Eq. 6.11 is identical to Eq. 6.9, since the predictor step in Eq. 6.7 is simply 
the explicit Euler method. The second line in Eq. 6.11 is obtained by noting that 
sa = F(tnsi,tr +h) 
= Nag Paes!) (6.12) 


Now we need to develop techniques for analyzing these difference equations so that 
we can compare the merits of the time-marching methods that generated them. 


6.3 Solution of Linear OAE’s With Constant Co- 
efficients 


The techniques for solving linear difference equations with constant coefficients is as 
well developed as that for ODE’s and the theory follows a remarkably parallel path. 
This is demonstrated by repeating some of the developments in Section 4.2, but for 
difference rather than differential equations. 
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6.3.1  First- and Second-Order Difference Equations 
First-Order Equations 
The simplest nonhomogeneous ОЛЕ of interest is given by the single first-order equa- 
tion 

Un41 = gus + ab” (6.13) 
where с, a, and b are, in general, complex parameters. The independent variable is 


now n rather than t, and since the equations are linear and have constant coefficients, 
c is not a function of either n or u. The exact solution of Eq. 6.13 is 


ab” 





Un = сүс? + 
р-а 


where c, is а constant determined by the initial conditions. In terms of the initial 
value of u it can be written 





Just as in the development of Eq. 4.10, one can readily show that the solution of the 
defective case, (b — c), 
Маі = CU, + ac" 
is 
иһ = [uo «ano l|g^ 


This can all be easily verified by substitution. 


Second-Order Equations 
The homogeneous form of a second-order difference equation is given by 
Uns + G1U5 41 + Gol, = 0 (6.14) 


Instead of the differential operator D = 4 used for ODE’s, we use for OAE’s the 


difference operator Æ (commonly referred to as the displacement or shift operator) 
and defined formally by the relations 
Un+1 = Eun, Un+k = E* un 


Further notice that the displacement operator also applies to exponents, thus 


Бо. р" = "е = Бо. pr 
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where o can be any fraction or irrational number. 

The roles of D and E are the same insofar as once they have been introduced to 
the basic equations the value of u(t) or un can be factored out. Thus Eq. 6.14 can 
now be re-expressed in an operational notion as 


(E? +a E + ag)u, = 0 (6.15) 


which must be zero for all un. Eq. 6.15 is known as the operational form of Eq. 6.14. 
The operational form contains a characteristic polynomial P(E) which plays the same 
role for difference equations that P(D) played for differential equations; that is, its 
roots determine the solution to the OAE. In the analysis of OAE’s, we label these 
roots 01, 02, -- ·, etc, and refer to them as the c-roots. They are found by solving the 
equation P(o) = 0. In the simple example given above, there are just two с roots 
and in terms of them the solution can be written 


Un = calo)” + сә(сә)" (6.16) 
where c; and c» depend upon the initial conditions. The fact that Eq. 6.16 is a 
solution to Eq. 6.14 for all с, c? and n should be verified by substitution. 
6.3.2 Special Cases of Coupled First-Order Equations 


A Complete System 


Coupled, first-order, linear homogeneous difference equations have the form 





2 am = cuf” ЗЕ cul” 
uft” = соди") + Co us? (6.17) 


which can also be written 


^ cud > F (n) (ny) _ |с C12 
Uni =, Cun Й Un = б ‚Мә , C 
Сә C22 


The operational form of Eq. 6.17 can be written 


u, 10 


(cu — E) C12 | ta |" -[e- E Thin =0 


C21 (c22 E E) 


which must be zero for all uj and uo. Again we are led to a characteristic polynomial, 
this time having the form P(E) = det|C — E I]. The o-roots are found from 





P(c) = det | n В | -0 


C21 (C29 = c) 
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Obviously the c; are the eigenvalues of C and, following the logic of Section 4.2, 
if x are its eigenvectors, the solution of Eq. 6.17 is 


ә 


2 
Un = 5 CklOk) Le 

k=1 
where ск are constants determined by the initial conditions. 


A. Defective System 


The solution of OAE’s with defective eigensystems follows closely the logic in Section 
4.2.2 for defective ODE’s. For example, one can show that the solution to 





Un+1 o Un 
Ün- 1| = 1 c tn 
Un+1 1 a] | up, 
is 
Un = с 
in. = [йо + допо Jo” 
x . n(n—1) . 
Un = E + Мопс ka D, | o” (6.18) 


6.4 Solution of the Representative OAE's 


6.4.1 The Operational Form and its Solution 


Examples of the nonhomogeneous, linear, first-order ordinary difference equations, 
produced by applying a time-marching method to the representative equation, are 
given by Eqs. 6.9 to 6.11. Using the displacement operator, E, these equations сап 
be written 


[E — (1+Ah)]un = h- ае" (6.19) 
[(1 — Ah) EF — 1]u, = h - E - ac^ (6.20) 

E —(1 + Аһ) ü 1 phn 
dome 5.3 lal [ie] ma 
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All three of these equations are subsets of the operational form of the representative 
OAE 


P(E)u, = Q(E) - ае!" (6.22) 


which is produced by applying time-marching methods to the representative ODE, Eq. 
4.45. We can express in terms of Eq. 6.22 all manner of standard time-marching meth- 
ods having multiple time steps and various types of intermediate predictor-corrector 
families. The terms P(E) and Q(E) are polynomials in E referred to as the charac- 
teristic polynomial and the particular polynomial, respectively. 

The general solution of Eq. 6.22 can be expressed as 


Q(e"^) 


Pe) (6.23) 


K 
Un = у> Chlap)” + ae” . 
k=1 


where c; аге ће K roots of the characteristic polynomial, P(o) = 0. When determi- 
nants are involved in the construction of P(E) and Q(E), as would be the case for 
Eq. 6.21, the ratio Q(E)/P(E) can be found by Kramer's rule. Keep in mind that 
for methods such as in Eq. 6.21 there are multiple (two in this case) solutions, one 
for un and à, and we are usually only interested in the final solution ил. Notice also, 
the important subset of this solution which occurs when и = 0, representing a time 
invariant particular solution, or a steady state. In such a case 





6.4.2 Examples of Solutions to Time-Marching OAE's 


As examples of the use of Eqs. 6.22 and 6.23, we derive the solutions of Eqs. 6.19 to 
6.21. For the explicit Euler method, Eq. 6.19, we have 


P(E) = Е-1- Аһ 


QUE) = h (6.24) 
and the solution of its representative OAE follows immediately from Eq. 6.23: 
Un = с\(1 + АҺ)" + ae” . ОБЕГ C 
И eh —1— АҺ 


For the implicit Euler method, Eq. 6.20, we have 


P(E) = (1—АҺ)Е—1 
О(Е) = ҺЕ (6.25) 
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SO 
Һе!" 


ак ct s 
a 1— АҺ (1 Аһ)е — 1 


In the case of the coupled predictor-corrector equations, Eq. 6.21, one solves for the 
final family u, (one can also find a solution for the intermediate family à), and there 
results 


E -(1-- AR) 


P(E) — det 
SE) de к Had 


| = Е(Е —1—һ— zo) 
THEM | да+ в T 


The o-root is found from 


1 
= ;ҺЕ(Е  1- Ah) 


1 
Dos (o desig 5л”) =0 


which has only one nontrivial root (c = 0 is simply a shift in the reference index). 
The complete solution can therefore be written 


1 
1 n 5h(e^ +1+ AR) 
TEE (1 + ye 5А) + ae^^n . 2 


———————— (6.26) 
e” —1—Ah— 5л 


6.5 The А – с Relation 


6.5.1 Establishing the Relation 


We have now been introduced to two basic kinds of roots, the A-roots and the c-roots. 
The former are the eigenvalues of the A matrix in the ODE's found by space differ- 
encing the original PDE, and the latter are the roots of the characteristic polynomial 
in a representative OAE found by applying a time-marching method to the repre- 
sentative ODE. There is a fundamental relation between the two which can be used 
to identify many of the essential properties of a time-march method. This relation is 
first demonstrated by developing it for the explicit Euler method. 

First we make use of the semi-discrete approach to find a system of ODE's and 
then express its solution in the form of Eq. 4.27. Remembering that t = nh, one can 
write 


u(t) = a (e) zi res (е) mu eulet)" ты + PS. (627) 
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where for the present we are not interested in the form of the particular solution 
(P.S.). Now the explicit Euler method produces for each A-root, one o-root, which 
is given by с = 1 + Ah. So if we use the Euler method for the time advance of the 
ODE's, the solution? of the resulting OAE is 


Un = clo)” 1i кәр а)" Lm sR eee см(см)" TM + PS. (6.28) 


where the c,, and the Lin in the two equations are identical and om = (1+ AmA). 
Comparing Eq. 6.27 and Eq. 6.28, we see a correspondence between om and e^"^, 
Since the value of е^^ can be expressed in terms of the series 


1 1 1 
eh = 1-„ AN + AR? pes eeepc APP Eee 
2 6 n! 


the truncated expansion с = 1 + Ah is a reasonable? approximation for small enough 
АР. 

Suppose, instead of the Euler method, we use the leapfrog method for the time 
advance, which is defined by 


Uni = Un- + 2hw, (6.29) 


Applying Eq. 6.8 to Eq. 6.29, we have the characteristic polynomial P(E) = E? — 


2AhE — 1, so that for every А the с must satisfy the relation 
оў, —2mhom — 1 = 0 (6.30) 


Now we notice that each А produces two c-roots. For one of these we find 


Amh d 1 + A2? (6.31) 


1 1 
1+A,, + g^» — Ат Tee (6.32) 


Om 


This is an approximation to eò” with an error O(A%h?). The other root, A,, — 


\/1 + A2,h?, will be discussed in Section 6.5.3. 


?Based on Section 4.4. 
?The error is О(А?А?). 
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6.5.2 The Principal c-Root 


Based on the above we make the following observation: 


Application of the same time-marching method to all of the equations in 

a coupled system linear ODE's in the form of Eq. 4.6, always produces 

one o-root for every A-root that satisfies the relation 

(6.33) 


= 105,9 1 kpk k+1 
0 — 1 Mor VW + + AN  O(n**1) 


where Ё is the order of the time-marching method. 





We refer to the root that has the above property as the principal a-root, and designate 
it (Om)1. The above property can be stated regardless of the details of the time- 
marching method, knowing only that its leading error is ошл Thus the principal 


root is an approximation to eò up to O(n‘). 
Note that a second-order approximation to a derivative written in the form 
(би) = (иһ — Un—1) (6.34) 


has a leading truncation error which is O(A?), while the second-order time-marching 
method which results from this approximation, which is the leapfrog method: 


ил-+1 = иһ—1 + 2ш, (6.35) 


has a leading truncation error O(h). This arises simply because of our notation 
for the time-marching method in which we have multiplied through by h to get an 
approximation for the function un+ı rather than the derivative as in Eq. 6.34. The 
following example makes this clear. Consider a solution obtained at a given time T' 
using a second-order time-marching method with a time step h. Now consider the 
solution obtained using the same method with a time step h/2. Since the error per 
time step is O(h3), this is reduced by a factor of eight (considering the leading term 
only). However, twice as many time steps are required to reach the time T. Therefore 
the error at the end of the simulation is reduced by a factor of four, consistent with 
a second-order approximation. 


6.5.3 Spurious c-Roots 


We saw from Eq. 6.30 that the А — с relation for the leapfrog method produces two 
c-roots for each А. One of these we identified as the principal root which always 
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has the property given in 6.33. The other is referred to as a spurious o-root and 
designated (om)2. In general, the А — c relation produced by a time-marching scheme 
can result in multiple o-roots all of which, except for the principal one, are spurious. 
All spurious roots are designated (om), where k = 2,3,---. No matter whether а 
o-root is principal or spurious, it is always some algebraic function of the product Ah. 
To express this fact we use the notation о = o (Ah). 

If a time-marching method produces spurious о-гооёѕ, the solution for ће ОЛЕ in 
the form shown in Eq. 6.28 must be modified. Following again the message of Section 
4.4, we have 


Un = сп (01)! Ti deese Сл (©т.)1 Tm pee смі(см)! IM + P.S. 


+¢12(01)5 Ly ae eer Ста (Ст) Lm кыа смә(© м)» Tm 


+¢13(01)3 i spes Ciel Oa). Lm ах смз(ом)з їм 


+etc., if there аге more spurious roots (6.36) 


Spurious roots arise if a method uses data from time level n — 1 or earlier to 
advance the solution from time level n to n 4- 1. Such roots originate entirely from 
the numerical approximation of the time-marching method and have nothing to do 
with the ODE being solved. However, generation of spurious roots does not, in itself, 
make a method inferior. In fact, many very accurate methods in practical use for 
integrating some forms of ODE's have spurious roots. 

It should be mentioned that methods with spurious roots are not self starting. 
For example, if there is one spurious root to a method, all of the coefficients (c)» 
in Eq. 6.36 must be initialized by some starting procedure. The initial vector 10 
does not provide enough data to initialize all of the coefficients. This results because 
methods which produce spurious roots require data from time level n — 1 or earlier. 
For example, the leapfrog method requires t,_; and thus cannot be started using 
only tin. 

Presumably (i.e., if one starts the method properly) the spurious coefficients are 
all initialized with very small magnitudes, and presumably the magnitudes of the 
spurious roots themselves are all less than one (see Chapter 7). Then the presence of 
spurious roots does not contaminate the answer. That is, after some finite time the 
amplitude of the error associated with the spurious roots is even smaller then when 
it was initialized. Thus while spurious roots must be considered in stability analysis, 
they play virtually no role in accuracy analysis. 


6.5.4 One-Root Time-Marching Methods 


There are a number of time-marching methods that produce only one o-root for each 
A-root. We refer to them as one-root methods. They are also called one-step methods. 
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They have the significant advantage of being self-starting which carries with it the 
very useful property that the time-step interval can be changed at will throughout 
the marching process. Three one-root methods were analyzed in Section 6.4.2. A 
popular method having this property, the so-called 6-method, is given by the formula 


Unt = Un + һа =Й ш + 6u, ы 


The 6-method represents the explicit Euler (0 = 0), the trapezoidal (0 = 2), and the 
implicit Euler methods (0 = 1), respectively. Its А — c relation is 


| 1+(1-6)Ah 
||. 1—8Ah 


It is instructive to compare the exact solution to a set of ODE's (with a complete 
eigensystem) having time-invariant forcing terms with the exact solution to the OAE’s 
for one-root methods. These are 

u(t) = afet)" ty +--+ es (et) Emt euft)" ty + Alf 
us coy wipe eto eyo cuts 5f 637) 
respectively. Notice that when ¢ and n = 0, these equations are identical, so that all 
the constants, vectors, and matrices are identical except the u and the terms inside 
the parentheses on the right hand sides. The only error made by introducing the time 
marching is the error that с makes in approximating е^". 


6.6 Accuracy Measures of Time-Marching Meth- 
ods 


6.6.1 Local and Global Error Measures 


There are two broad categories of errors that can be used to derive and evaluate time- 
marching methods. One is the error made in each time step. This is a local error such 
as that found from a Taylor table analysis, see Section 3.4. It is usually used as the 
basis for establishing the order of a method. The other is the error determined at the 
end of a given event which has covered a specific interval of time composed of many 
time steps. This is a global error. It is useful for comparing methods, as we shall see 
in Chapter 8. 

It is quite common to judge a time-marching method on the basis of results found 
from a Taylor table. However, a Taylor series analysis is a very limited tool for finding 
the more subtle properties of a numerical time-marching method. For example, it is 
of no use in: 
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finding spurious roots. 


evaluating numerical stability and separating the errors in phase and amplitude. 


e analyzing the particular solution of predictor-corrector combinations. 


finding the global error. 


The latter three of these are of concern to us here, and to study them we make use of 
the material developed in the previous sections of this chapter. Our error measures 
are based on the difference between the exact solution to the representative ODE, 
given by 


ac"! 


t) = се% 
u(t) = ce iE 


(6.38) 


and the solution to the representative OAE's, including only the contribution from 
the principal root, which can be written as 


Un = С1(01)" + ae^^ . 


(6.39) 


6.6.2 Local Accuracy of the Transient Solution (ery, |o| , eru) 
Transient error 


The particular choice of an error measure, either local or global, is to some extent 
arbitrary. However, a necessary condition for the choice should be that the measure 
can be used consistently for all methods. In the discussion of the A-c relation we 
saw that all time-marching methods produce a principal o-root for every A-root that 
exists in a set of linear ODE’s. Therefore, a very natural local error measure for the 
transient solution is the value of the difference between solutions based on these two 
roots. We designate this by er, and make the following definition 


егу = e — 


01 

The leading error term can be found by expanding in a Taylor series and choosing 
the first nonvanishing term. This is similar to the error found from a Taylor table. 
The order of the method is the last power of Ah matched exactly. 
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Amplitude and Phase Error 


Suppose a À eigenvalue is imaginary. Such can indeed be the case when we study the 
equations governing periodic convection which produces harmonic motion. For such 
cases it is more meaningful to express the error in terms of amplitude and phase. 
Let А = iw where w is a real number representing a frequency. Then the numerical 
method must produce a principal o-root that is complex and expressible in the form 


о = 0, + do; а ei^ (6.40) 


From this it follows that the local error in amplitude is measured by the deviation of 
[c1| from unity, that is 





and the local error in phase can be defined as 
er, = wh — tan [(01);/(o1)r)| (6.41) 


Amplitude and phase errors are important measures of the suitability of time-marching 
methods for convection and wave propagation phenomena. 

The approach to error analysis described in Section 3.5 can be extended to the 
combination of a spatial discretization and a time-marching method applied to the 
linear convection equation. The principal root, оу (Ал), is found using A = —iak*, 
where к* is the modified wavenumber of the spatial discretization. Introducing the 
Courant number, C, = ah/Az, we have Ah = —їС„к*Ах. Thus one can obtain 
values of the principal root over the range 0 < &Ax < т for a given value of the 
Courant number. The above expression for er, can be normalized to give the error 
in the phase speed, as follows 


его tan ^ [(o1)i/(o1)r)] 
= 29 1+ 6.42 
i wh С„кАх ) 
where ш = —ak. A positive value of er, corresponds to phase lag (the numerical phase 


speed is too small), while a negative value corresponds to phase lead (the numerical 
phase speed is too large). 


6.6.3 Local Accuracy of the Particular Solution (er,,) 
The numerical error in the particular solution is found by comparing the particular 
solution of the ODE with that for the OAE. We have found these to be given by 

1 


P.S. = get . — 
ds M = 
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and 

Q(e^^) 

Pg 

respectively. For a measure of the local error in the particular solution we introduce 
the definition 





P.S.toAE) = ac" Ў 


г i} (6.43) 


The multiplication by h converts the error from a global measure to a local one, so 
that the order of er, and er, are consistent. In order to determine the leading error 
term, Eq. 6.43 can be written in terms of the characteristic and particular polynomials 
as 


eru = nes : {(u — X)Q(e^^) — P(e^)! (6.44) 
where 
Co = lim = 
һ—0 Р(е"") 


The value of с, is а method-dependent constant that is often equal to one. If the 
forcing function is independent of time, u is equal to zero, and for this case, many 
numerical methods generate an ег, that is also zero. 

The algebra involved in finding the order of er, can be quite tedious. However, 
this order is quite important in determining the true order of a time-marching method 
by the process that has been outlined. An illustration of this is given in the section 
on Runge-Kutta methods. 


6.6.4 Time Accuracy For Nonlinear Applications 


In practice, time-marching methods are usually applied to nonlinear ODE's, and it 
is necessary that the advertised order of accuracy be valid for the nonlinear cases as 
well as for the linear ones. А necessary condition for this to occur is that the local 
accuracies of both the transient and the particular solutions be of the same order. 
More precisely, a time-marching method is said to be of order k if 


er, = е - (АА)! (6.45) 


ea en) (6.46) 
where К = smallest о, Ко) (6.47) 
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The reader should be aware that this is not sufficient. For example, to derive all of 
the necessary conditions for the fourth-order Runge-Kutta method presented later 
in this chapter the derivation must be performed for a nonlinear ODE. However, the 
analysis based on a linear nonhomogeneous ODE produces the appropriate conditions 
for the majority of time-marching methods used in CFD. 


6.6.5 Global Accuracy 


In contrast to the local error measures which have just been discussed, we can also 
define global error measures. These are useful when we come to the evaluation of 
time-marching methods for specific purposes. This subject is covered in Chapter 8 
after our introduction to stability in Chapter 7. 

Suppose we wish to compute some time-accurate phenomenon over a fixed interval 
of time using a constant time step. We refer to such a computation as an "event". 
Let T' be the fixed time of the event and h be the chosen step size. Then the required 
number of time steps, is N, given by the relation 


T = Nh 


Global error in the transient 


A natural extension of er, to cover the error in an entire event is given by 


Er, = e — (o(Ah))" (6.48) 


Global error in amplitude and phase 


If the event is periodic, we are more concerned with the global error in amplitude and 
phase. These are given by 


Er, =1- ( (as (03). (6.49) 


and 





Er, = fun — tant (5) 


wT — N tan ! [(o1);/(o1)r] (6.50) 
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Global error in the particular solution 


Finally, the global error in the particular solution follows naturally by comparing the 
solutions to the ODE and the OAE. It can be measured by 





6.7 Linear Multistep Methods 


In the previous sections, we have developed the framework of error analysis for time 
advance methods and have randomly introduced a few methods without addressing 
motivational, developmental or design issues. In the subsequent sections, we introduce 
classes of methods along with their associated error analysis. We shall not spend much 
time on development or design of these methods, since most of them have historic 
origins from a wide variety of disciplines and applications. The Linear Multistep 
Methods (LMM’s) are probably the most natural extension to time marching of the 
space differencing schemes introduced in Chapter 3 and can be analyzed for accuracy 
or designed using the Taylor table approach of Section 3.4. 


6.7.1 The General Formulation 
When applied to the nonlinear ODE 


d 
T = u ced) 


all linear multistep methods can be expressed in the general form 


1 1 
5 QkUn+k = h 5 BeF nik (6.51) 
=1— 


k K k=1-K 


where the notation for F is defined in Section 6.1. The methods are said to be linear 
because the a’s and 2° are independent of u and n, and they are said to be K-step 
because K time-levels of data are required to marching the solution one time-step, h. 
They are explicit if 6, = 0 and implicit otherwise. 

When Eq. 6.51 is applied to the representative equation, Eq. 6.8, and the result is 
expressed in operational form, one finds 


k=1-K 


| 5 2 иһ = { У а) (Аи, + ae”) (6.59) 
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We recall from Section 6.5.2 that a time-marching method when applied to the repre- 
sentative equation must provide a c-root, labeled с, that approximates e^ through 
the order of the method. The condition referred to as consistency simply means that 
о — 1 as h — 0, and it is certainly a necessary condition for the accuracy of any 
time marching method. We can also agree that, to be of any value in time accuracy, 
a method should at least be first-order accurate, that is с > (1 + AA) as h > 0. One 
can show that these conditions are met by any method represented by Eq. 6.51 if 


pO Lu and УВ. = M (K +k - lay 
k k k 


Since both sides of Eq. 6.51 can be multiplied by an arbitrary constant, these methods 
are often “normalized” by requiring 


У 8 =1 
k 


Under this condition c, = 1 in Eq. 6.44. 


6.7.2 Examples 


There are many special explicit and implicit forms of linear multistep methods. Two 
well-known families of them, referred to as Adams-Bashforth (explicit) and Adams- 
Moulton (implicit), can be designed using the Taylor table approach of Section 3.4. 
The Adams-Moulton family is obtained from Eq. 6.51 with 


a, — 1, оу = —1, ок = 0, К=—1,—2,-.- (6.53) 
The Adams-Bashforth family has the same a’s with the additional constraint that 
B, = 0. The three-step Adams-Moulton method can be written in the following form 

Uni = Un + (Bus үү + Pot, + 8-10. 1 + 8-20, 3) (6.54) 
A Taylor table for Eq. 6.54 can be generated as 


Un+1 

EA 

—hf ul, y -fi -fi -64 -b$ 
= Вои, — bo 
8101 —8-1 B-1 -b4 Bid 
—һ@-эш„_» -(-298., —(—2)!8› —(—2)98—›9 —(—2)98-›6 


Table 6.1. Taylor table for the Adams-Moulton three-step linear multistep method. 
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This leads to the linear system 


11 Е 
2 0 -2 —-4||% 
30 3 12|] ae 
4 0—4 -32 || 8 


(6.55) 


[qum Lm 


to solve for the "s, resulting in 
Bi =9/24, В —19/24,  8.,——5/24, | 8.9 — 1/24 (6.56) 


which produces a method which is fourth-order accurate.* With 3, = 0 one obtains 
EN Ж SN 


КЕ е 


Oo = 23/12, л ==16/12, $525/12 (6.58) 


This is the third-order Adams-Bashforth method. 

A list of simple methods, some of which are very common in CFD applications, 
is given below together with identifying names that are sometimes associated with 
them. In the following material AB(n) and AM(n) are used as abbreviations for the 
(n)th order Adams-Bashforth and (n)th order Adams-Moulton methods. One can 
verify that the Adams type schemes given below satisfy Eqs. 6.55 and 6.57 up to the 
order of the method. 


giving 


Explicit Methods 





pp = Un + hu, Euler 
Un41 = Un_1 + 2ш, Leapfrog 
pp = Un + In[u, — uha AB2 
Unt = Un + 523и, — 16u, , + 5и ;| АВЗ 


Implicit Methods 





Unyi = Un + һи Implicit Euler 
Шар = Un + In[w, + TAN Trapezoidal (AM2) 
spi = 1и, — un-1 + 2ш, a 2nd-order Backward 
Ur = Un + $5 ET, + 8u, — ТЕ АМЗ 


^Recall from Section 6.5.2 that a kth-order time-marching method has a leading truncation error 
term which is O(h**1). 
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6.7.3 Two-Step Linear Multistep Methods 


High resolution CFD problems usually require very large data sets to store the spatial 
information from which the time derivative is calculated. This limits the interest 
in multistep methods to about two time levels. The most general two-step linear 
multistep method (i.e., К=2 in Eq. 6.51), that is at least first-order accurate, can be 
written as 


(1+ Eunga = [(1 + 2£)us — £u, 1] + h [Bu + (1-0 +) ш, — puni] (6.59) 


Clearly the methods are explicit if 0 — 0 and implicit otherwise. A list of methods 
contained in Eq. 6.59 is given in Table 6.2. Notice that the Adams methods have 
€ = 0, which corresponds to оу = 0 in Eq. 6.51. Methods with € = —1/2, which 
corresponds to ао = 0 in Eq. 6.51, are known as Milne methods. 


Method 


Euler 
Implicit Euler 
Trapezoidal or AM2 
2nd Order Backward 
Adams type 
Lees Type 
Two-step trapezoidal 
A-contractive 
Leapfrog 
AB2 
Most accurate explicit 
'Third-order implicit 
AM3 
Milne 


H- Q2 C9 Q2 м b2 b32 М2 М2 b2 h2 МЮ —— EH 


Table 6.2. Some linear one- and two-step methods, see Eq. 6.59. 





One can show after a little algebra that both er, and егу are reduced to 0(h?) 
(i.e., the methods are 2nd-order accurate) if 


1 
ж а 


The class of all 3rd-order methods is determined by imposing the additional constraint 


5 
= 20 — > 
Scu enc 


Finally a unique fourth-order method is found by setting 0 = —y = —£/3 = Ü 
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6.8 Predictor-Corrector Methods 


There are a wide variety of predictor-corrector schemes created and used for a variety 
of purposes. Their use in solving ODE’s is relatively easy to illustrate and understand. 
Their use in solving PDE's can be much more subtle and demands concepts? which 
have no counterpart in the analysis of ODE's. 

Predictor-corrector methods constructed to time-march linear or nonlinear ODE’s 
are composed of sequences of linear multistep methods, each of which is referred to 
as a family in the solution process. There may be many families in the sequence, and 
usually the final family has a higher Taylor-series order of accuracy than the inter- 
mediate ones. Their use is motivated by ease of application and increased efficiency, 
where measures of efficiency are discussed in the next two chapters. 

А simple one-predictor, one-corrector example is given by 


ügpa = Un + ahu, 

Unyi = Un +h] Bü, + уш, (6.60) 
where the parameters о, 8 and y are arbitrary parameters to be determined. One 
can analyze this sequence by applying it to the representative equation and using 
the operational techniques outlined in Section 6.4. It is easy to show, following the 
example leading to Eq. 6.26, that 

P(E) = E*-|E -1— (y+ 8h — apah] (6.61) 
Q(E) = E?*-h-|8E? + y + aBAR] (6.62) 
Considering only local accuracy, one is led, by following the discussion in Section 6.6, 


to the following observations. For the method to be second-order accurate both ery 
and er, must be O(h?). For this to hold for егу, it is obvious from Eq. 6.61 that 


1 
yt+B=1 ; ад = 5 


which provides two equations for three unknowns. The situation for ег, requires some 
algebra, but it is not difficult to show using Eq. 6.44 that the same conditions also 
make it O(h?). One concludes, therefore, that the predictor-corrector sequence 


iad 1 
нао = Un + ahu, 


Quern em 


is a second-order accurate method for any o. 








1 
Un+1 = Un + sh 


5Such as alternating direction, fractional-step, and hybrid methods. 
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А classical predictor-corrector sequence is formed by following an Adams-Bashforth 
predictor of any order with an Adams-Moulton corrector having an order one higher. 
The order of the combination is then equal to the order of the corrector. If the order 
of the corrector is (k), we refer to these as ABM(k) methods. The Adams-Bashforth- 
Moulton sequence for k = 3 is 


Un41 n—1 


Un + =з, — u! | 





ces 
Шм = Unt 75 ЕА + 8ш, — usi] (6.64) 


Some simple, specific, second-order accurate methods are given below. The Gazdag 
method, which we discuss in Chapter 8, is 


Пада = Unt EA = TA 





1 
TN E es shit, tu (6.65) 
The Burstein method, obtained from Eq. 6.63 with o — 1/2 is 
Lo 
Ün+1/2 = Unt 5л 


and, finally, MacCormack's method, presented earlier in this chapter, is 


d ГА 
йн = Unt hu, 





1 А " 
Unyi = z lun + ting + hs (6.67) 
Note that MacCormack’s method can also be written as 


m ГА 
Ши = Un + hup 





1 
dog = Un + Plun + 1] (6.68) 


from which it is clear that it is obtained from Eq. 6.63 with a = 1. 


6.9 Runge-Kutta Methods 
There is a special subset of predictor-corrector methods, referred to as Runge-Kutta 
methods, that produce just one o-root for each A-root such that o(Ah) corresponds 


ê Although implicit and multi-step Runge-Kutta methods exist, we will consider only single-step, 
explicit Runge-Kutta methods here. 
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to the Taylor series expansion of eò” out through the order of the method and then 
truncates. Thus for a Runge-Kutta method of order Ё (up to 4th order), the principal 
(and only) o-root is given by 


1 1 
o=1+)\h+ j^ Ted д^ (6.69) 
It is not particularly difficult to build this property into a method, but, as we pointed 
out in Section 6.6.4, it is not sufficient to guarantee k’th order accuracy for the solution 
of и = F(u,t) or for the representative equation. To ensure k’th order accuracy, the 
method must further satisfy the constraint that 


er, = O(h**!) (6.70) 


and this is much more difficult. 

The most widely publicized Runge-Kutta process is the one that leads to the 
fourth-order method. We present it below in some detail. It is usually introduced in 
the form 


ky = hF(un,tn) 
ko = hF (un + ki, t, + ah) 
Ёз = ҺЕ(и„ + Byky + ^y Ko, tn + œh) 
k4 = hF (un + Bok, + yok + d2k3, tn + azh) 
followed by 
u(tn + h) = u(tn) = pa ka + bok + изз + Ita ka (6.71) 


However, we prefer to present it using predictor-corrector notation. Thus, a scheme 
entirely equivalent to 6.71 is 








tage = Un + bhul, 
й+у = Unt Bihun + yha ү, 
Unies = Un + һи + hinta + бо cu 

Unyi = Unt phun + ah, у „+ из, о + ШАРТ, Le, (6.72) 


Appearing in Eqs. 6.71 and 6.72 are a total of 13 parameters which are to be 
determined such that the method is fourth-order according to the requirements in 
Eqs. 6.69 and 6.70. First of all, the choices for the time samplings, a, ол, and o», are 
not arbitrary. They must satisfy the relations 


а.ч. 
O1 = + 
о = В + үә + 62 (6.73) 
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The algebra involved in finding algebraic equations for the remaining 10 parameters 
is not trivial, but the equations follow directly from finding P(E) and Q(E) and then 
satisfying the conditions in Eqs. 6.69 and 6.70. Using Eq. 6.73 to eliminate the 0s 
we find from Eq. 6.69 the four conditions 


pa + pa + из + pa = 1 (1) 
оо + изо + шод = 1/2 (2) (6.74) 
Hay + pa(ay + обо) = 1/6 (3) | 
1407109 = 1/24 (4) 


These four relations guarantee that the five terms in о exactly match the first 5 terms 
in the expansion of е^". To satisfy the condition that er, = O(k?), we have to fulfill 
four more conditions 


изо? + uisa] + H403 = 1/3 (3) 
Шао? + igo + p403 = 1/4 (4) (6.75) 
uso? + ш(а?т + ад) = 1/12 (4) | 
изо ү. + Шао (аә + олдо) = 1/8 (4) 


The number in parentheses at the end of each equation indicates the order that 
is the basis for the equation. Thus if the first 3 equations in 6.74 and the first 
equation in 6.75 are all satisfied, the resulting method would be third-order accurate. 
As discussed in Section 6.6.4, the fourth condition in Eq. 6.75 cannot be derived 
using the methodology presented here, which is based on a linear nonhomogenous 
representative ODE. A more general derivation based on a nonlinear ODE can be 
found in several books." 

There are eight equations in 6.74 and 6.75 which must be satisfied by the 10 
unknowns. Since the equations are overdetermined, two parameters can be set arbi- 
trarily. Several choices for the parameters have been proposed, but the most popular 
one is due to Runge. It results in the “standard” fourth-order Runge-Kutta method 
expressed in predictor-corrector form as 








7 1 
Un 41/2 = Unt 51 
Üüs41/2 = Unt 5 tn t1/2 
1541 = Un + hj 41/2 
1 1 Al ~I — 
Unyi = CE chis. + ой + ics) 4 7, (6.76) 


"The present approach based on a linear inhomogeneous equation provides all of the necessary 
conditions for Runge-Kutta methods of up to third order. 
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Notice that this represents the simple sequence of conventional linear multistep meth- 
ods referred to, respectively, as 


Euler Predictor 
Euler Corrector 
Leapfrog Predictor 
Milne Corrector 


— RKA 


One can easily show that both the Burstein and the MacCormack methods given by 
Eqs. 6.66 and 6.67 are second-order Runge-Kutta methods, and third-order methods 
can be derived from Eqs. 6.72 by setting u4 = 0 and satisfying only Eqs. 6.74 and the 
first equation in 6.75. It is clear that for orders one through four, RK methods of order 
k require k evaluations of the derivative function to advance the solution one time 
step. We shall discuss the consequences of this in Chapter 8. Higher-order Runge- 
Kutta methods can be developed, but they require more derivative evaluations than 
their order. For example, a fifth-order method requires six evaluations to advance 
the solution one step. In any event, storage requirements reduce the usefulness of 
Runge-Kutta methods of order higher than four for CFD applications. 


6.10 Implementation of Implicit Methods 


We have presented a wide variety of time-marching methods and shown how to derive 
their А — c relations. In the next chapter, we will see that these methods can have 
widely different properties with respect to stability. 'This leads to various trade- 
offs which must be considered in selecting a method for a specific application. Our 
presentation of the time-marching methods in the context of a linear scalar equation 
obscures some of the issues involved in implementing an implicit method for systems 
of equations and nonlinear equations. These are covered in this Section. 


6.10.1 Application to Systems of Equations 


Consider first the numerical solution of our representative ODE 
и! = Аи + ае! (6.77) 


using the implicit Euler method. Following the steps outlined in Section 6.2, we 
obtained 


(1— АА)шь ці — Un = het” . ae” (6.78) 
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Solving for uni, gives 
1 
= nh | phn 
Unti = тур gn + ће. gen) (6.79) 


This calculation does not seem particularly onerous in comparison with the applica- 
tion of an explicit method to this ODE, requiring only an additional division. 

Now let us apply the implicit Euler method to our generic system of equations 
given by 


й = Aŭ — f(t) (6.80) 


where 4/ and Ў are vectors and we still assume that А is not a function of u or t. Now 
the equivalent to Eq. 6.78 is 


> 


(I — hAyü, 1 — d, = —hf (t + h) (6.81) 
or 
йл = (1 ҺА) [ü, — hf (t + h)] (6.82) 


The inverse is not actually performed, but rather we solve Eq. 6.81 as a linear system 
of equations. For our one-dimensional examples, the system of equations which must 
be solved is tridiagonal (e.g., for biconvection, A = —aB,(—1,0, 1)/2Az), and hence 
its solution is inexpensive, but in multidimensions the bandwidth can be very large. In 
general, the cost per time step of an implicit method is larger than that of an explicit 
method. The primary area of application of implicit methods is in the solution of 
stiff ODE's, as we shall see in Chapter 8. 


6.10.2 Application to Nonlinear Equations 


Now consider the general nonlinear scalar ODE given by 


du 
— = F(u,t 6.83 
= = Fut) (6.83) 
Application of the implicit Euler method gives 

Un+1 = Un + AF (usa, tn41) (6.84) 


This is a nonlinear difference equation. As an example, consider the nonlinear ODE 


— +w = 0 (6.85) 
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solved using implicit Euler time marching, which gives 
1, 
Ungi F hts = иһ (6.86) 


which requires a nontrivial method to solve for из. There are several different 
approaches one can take to solving this nonlinear difference equation. An iterative 
method, such as Newton's method (see below), can be used. In practice, the "initial 
guess" for this nonlinear problem can be quite close to the solution, since the "initial 
guess" is simply the solution at the previous time step, which implies that a lineariza- 
tion approach may be quite successful. Such an approach is described in the next 
Section. 


6.10.3 Local Linearization for Scalar Equations 
General Development 


Let us start the process of local linearization by considering Eq. 6.83. In order to 
implement the linearization, we expand F(u,t) about some reference point in time. 
Designate the reference value by t, and the corresponding value of the dependent 
variable by u,. A Taylor series expansion about these reference quantities gives 





as) е ib ү (6.87) 


On the other hand, the expansion of u(t) in terms of the independent variable t is 


u(t) = Un + (t — tn) (27), + s(t Ec E Tee (6.88) 


If t is within h of t,, both (t — t,)^ and (и — un) are O(h*), and Eq. 6.87 can be 
written 


F(u,t) = Fp + (F) e — Un) + (F) e — tn) + О(Һ?) (6.89) 
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Notice that this is an expansion of the derivative of the function. Thus, relative to the 
order of expansion of the function, it represents a second-order-accurate, locally-linear 
approximation to F(u,t) that is valid in the vicinity of the reference station tn and 
the corresponding и, = u(t,). With this we obtain the locally (in the neighborhood 
of tn) time-linear representation of Eq. 6.83, namely 


m (E) « " (в z (25) =) А (F) е ta) + O(h?) (690) 


Implementation of the Trapezoidal Method 


As an example of how such an expansion can be used, consider the mechanics of 
applying the trapezoidal method for the time integration of Eq. 6.83. The trapezoidal 
method is given by 


1 
Unt1 = Un + УМК + Fh] + hO(nh?) (6.91) 


where we write hO(h?) to emphasize that the method is second order accurate. Using 
Eq. 6.89 to evaluate Ё, = F(u, 41,1541), one finds 


7 1 OF OF з 


+hO(h’) (6.92) 


Note that the O(h?) term within the brackets (which is due to the local linearization) 
is multiplied by h and therefore is the same order as the hO(h?) error from the 
Trapezoidal Method. The use of local time linearization updated at the end of each 
time step, and the trapezoidal time march, combine to make a second-order-accurate 
numerical integration process. There are, of course, other second-order implicit time- 
marching methods that can be used. The important point to be made here is that 
local linearization updated at each time step has not reduced the order of accuracy of 
а second-order time-marching process. 
А very useful reordering of the terms in Eq. 6.92 results in the expression 


1 (OF Delar 


which is now in the delta form which will be formally introduced in Section 12.6. In 
many fluid mechanic applications the nonlinear function F is not an explicit function 
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of t. In such cases the partial derivative of F(u) with respect to t is zero and Eq. 6.93 
simplifies to the second-order accurate expression 


1, (дЕ 
1— -h| — = .94 
o Jou an T 


Notice that the RHS is extremely simple. It is the product of h and the RHS of 
the basic equation evaluated at the previous time step. In this example, the basic 
equation was the simple scalar equation 6.83, but for our applications, it is generally 
the space-differenced form of the steady-state equation of some fluid flow problem. 

А numerical time-marching procedure using Eq. 6.94 is usually implemented as 
follows: 


1. Solve for the elements of AF,, store them in an array say Ё, and save tp. 


2. Solve for the elements of the matrix multiplying Aü, and store in some appro- 
priate manner making use of sparseness or bandedness of the matrix if possible. 
Let this storage area be referred to as В. 


3. Solve the coupled set of linear equations 
ВАй„ = Ё 
for Aii,. (Very seldom does one find B^! in carrying out this step). 
4. Find w,41 by adding Aw, to ün, thus 
Uns = Atl, + Up 


The solution for 44,44 is generally stored such that it overwrites the value of ti, 
and the process is repeated. 


Implementation of the Implicit Euler Method 
We have seen that the first-order implicit Euler method can be written 
Un+l = Un + ҺЕ (6.95) 


if we introduce Eq. 6.90 into this method, rearrange terms, and remove the explicit 
dependence on time, we arrive at the form 


ken || ao 
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We see that the only difference between the implementation of the trapezoidal method 
and the implicit Euler method is the factor of 1 іп the brackets of the left side of 
Eqs. 6.94 and 6.96. Omission of this factor degrades the method in time accuracy by 
one order of h. We shall see later that this method is an excellent choice for steady 
problems. 


Newton's Method 


Consider the limit А — oo of Eq. 6.96 obtained by dividing both sides by ^ and 
setting 1/h — 0. There results 


OF 
E | a) Au = F, (6.97) 
Or 
NM с” (6.98) 
Un+1 = Un ðu Ё n . 


This is the well-known Newton method for finding the roots of a nonlinear equation 
F(u) — 0. The fact that it has quadratic convergence is verified by a glance at Eqs. 
6.87 and 6.88 (remember the dependence on t has been eliminated for this case). By 
quadratic convergence, we mean that the error after a given iteration is proportional 
to the square of the error at the previous iteration, where the error is the difference 
between the current solution and the converged solution. Quadratic convergence is 
thus a very powerful property. Use of a finite value of h in Eq. 6.96 leads to linear 
convergence, i.e., the error at a given iteration is some multiple of the error at the 
previous iteration. The reader should ponder the meaning of letting h — oo for the 
trapezoidal method, given by Eq. 6.94. 


6.10.4 Local Linearization for Coupled Sets of Nonlinear Equa- 
tions 


In order to present this concept, let us consider an example involving some sim- 
ple boundary-layer equations. We choose the Falkner-Skan equations from classical 
boundary-layer theory. Our task is to apply the implicit trapezoidal method to the 


equations 
Qo uf df\*\ _ 
a ке +a(.- (E) ) =o Po 


Here f represents a dimensionless stream function, and f is a scaling factor. 
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First of all we reduce Eq. 6.99 to a set of first-order nonlinear equations by the 
transformations 


d’ f df 
= de иә = dp o? 
This gives the coupled set of three nonlinear equations 
и = F = –щиз 8 uj) 
иә = pod 
u = һ=и» (6.101) 


из = / (6.100) 


ui 





and these can be represented in vector notation as 


du 
К ЫШ F(u) (6.102) 

Now we seek to make the same local expansion that derived Eq. 6.90, except that 
this time we are faced with a nonlinear vector function, rather than a simple nonlinear 
scalar function. The required extension requires the evaluation of a matrix, called 
the Jacobian matrix.) Let us refer to this matrix as A. It is derived from Eq. 6.102 
by the following process 


For the general case involving a third order matrix this is 


OF, дЕ OF 
ди дио диз 


42125 9E 8E 


ди дш дщ 


ӘР, OF, BF 
Bn. 0s Ou. 


The expansion of F(u) about some reference state и, can be expressed in a way 
similar to the scalar expansion given by eq 6.87. Omitting the explicit dependency 


(6.104) 


on the independent variable t, and defining Ё„ as F (tn), one has ? 


5Recall that we derived the Jacobian matrices for the two-dimensional Euler equations in Section 
2.2 

?The Taylor series expansion of a vector contains a vector for the first term, a matrix times а 
vector for the second term, and tensor products for the terms of higher order. 
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F(t) = Fy + As(u — tn) + O(R?) (6.105) 


where t —t, and the argument for O(h?) is the same as in the derivation of Eq. 6.88. 
Using this we can write the local linearization of Eq. 6.102 as 


> 


~ = Ant + |А, - Амы) +O(h?) (6.106) 
— a 
“constant” 


which is a locally-linear, second-order-accurate approximation to a set of coupled 
nonlinear ordinary differential equations that is valid for t < tn + h. Any first- or 
second-order time-marching method, explicit or implicit, could be used to integrate 
the equations without loss in accuracy with respect to order. The number of times, 
and the manner in which, the terms in the Jacobian matrix are updated as the solution 
proceeds depends, of course, on the nature of the problem. 
Returning to our simple boundary-layer example, which is given by Eq. 6.101, we 
find the Jacobian matrix to be 
— из 2ßuz —u 
A= 1 0 0 (6.107) 
0 1 0 


The student should be able to derive results for this example that are equivalent to 
those given for the scalar case in Eq. 6.93. Thus for the Falkner-Skan equations the 
trapezoidal method results in 


1+ 20), —Bh(u2), (ш), | E (^ui), о best 
I 1 0 (ио) | Sh (ш), 
0 -& 1 (Aus)n (шо), 


We find ИА from Аш» + ün, and the solution is now advanced one step. Re-evaluate 
the elements using и, ут and continue. Without any iterating within a step advance, 
the solution will be second-order-accurate in time. 


6.11 Problems 


1. Find an expression for the nth term in the Fibonacci series, which is given 
Бу 1,1,2,3,5,8,... Note that the series can be expressed as the solution to a 
difference equation of the form из = Un + Un—1. What is u25? (Let the first 
term given above be up.) 
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2. The trapezoidal method и = Un + 3h(u,, + uj) is used to solve the repre- 
sentative ODE. 
(a) What is the resulting OAE? 
(b) What is its exact solution? 
(c) How does the exact steady-state solution of the OAE compare with the 
exact steady-state solution of the ODE if u = 0? 


3. The 2nd-order backward method is given by 


1 
Un+1 = 3 14и, — Ир 1 F 2 hu, д 


(a) Write the OAE for the representative equation. Identify the polynomials 
P(E) and Q(E). 


(b) Derive the A-c relation. Solve for the o-roots and identify them as principal 
ог spurious. 


c) Find er) and the first two nonvanishing terms in a Taylor series expansion 
8 p 
of the spurious root. 


(d) Perform a o-root trace relative to the unit circle for both diffusion and 


convection. 


4. Consider the time-marching scheme given by 


! ! ! 
Un+tl = Un—-1 + ni + Un + M ed) 


(a) Write the OAE for the representative equation. Identify the polynomials 
P(E) and Q(E). 


(b) Derive the А — relation. 
(c) Find er). 
5. Find the difference equation which results from applying the Gazdag predictor- 


corrector method (Eq. 6.65) to the representative equation. Find the A-o rela- 
tion. 


6. Consider the following time-marching method: 


Un+1/3 = Unt hu, [3 
Un+1/2 = Unt hitns1/3/2 


Until = Un + Валу 








6.11. 


10. 


11. 
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Find the difference equation which results from applying this method to the 
representative equation. Find the A-o relation. Find the solution to the differ- 
ence equation, including the homogeneous and particular solutions. Find ег, 
and er,. What order is the homogeneous solution? What order is the particular 
solution? Find the particular solution if the forcing term is fixed. 


. Write a computer program to solve the one-dimensional linear convection equa- 


tion with periodic boundary conditions and a = 1 on the domain 0 < х < 1. 
Use 2nd-order centered differences in space and a grid of 50 points. For the 
initial condition, use 


u(x, 0) = е795102—0.5)/0]? 


with с = 0.08. Use the explicit Euler, 2nd-order Adams-Bashforth (AB2), im- 
plicit Euler, trapezoidal, and 4th-order Runge-Kutta methods. For the explicit 
Euler and AB2 methods, use a Courant number, ah/ Ax, of 0.1; for the other 
methods, use a Courant number of unity. Plot the solutions obtained at t = 1 
compared to the exact solution (which is identical to the initial condition). 


. Repeat problem 7 using 4th-order (noncompact) differences in space. Use only 


4th-order Runge-Kutta time marching at a Courant number of unity. Show 
solutions at t = 1 and t = 10 compared to the exact solution. 


. Using the computer program written for problem 7, compute the solution at 


t = 1 using 2nd-order centered differences in space coupled with the 4th-order 
Runge-Kutta method for grids of 100, 200, and 400 nodes. On a log-log scale, 
plot the error given by 





exact : 


where M is the number of grid nodes and u is the exact solution. Find the 


global order of accuracy from the plot. 


Using the computer program written for problem 8, repeat problem 9 using 
Ath-order (noncompact) differences in space. 


Write a computer program to solve the one-dimensional linear convection equa- 
tion with inflow-outflow boundary conditions and a = 1 on the domain 0 < 
x <1. Let u(0,t) = sinwt with w = 107. Run until a periodic steady state 
is reached which is independent of the initial condition and plot your solution 
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12. 


13. 
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compared with the exact solution. Use 2nd-order centered differences in space 
with а Ist-order backward difference at the outflow boundary (as in Eq. 3.69) 
together with 4th-order Runge-Kutta time marching. Use grids with 100, 200, 
and 400 nodes and plot the error vs. the number of grid nodes, as described in 
problem 9. Find the global order of accuracy. 


Repeat problem 11 using 4th-order (noncompact) centered differences. Use a 
third-order forward-biased operator at the inflow boundary (as in Eq. 3.67). At 
the last grid node, derive and use a 3rd-order backward operator (using nodes 
j—3,j—2, j — 1, and j) and at the second last node, use a 3rd-order backward- 
biased operator (using nodes j — 2, j — 1, j, and j +1; see problem 1 in Chapter 
3). 


Using the approach described in Section 6.6.2, find the phase speed error, erp, 
and the amplitude error, era, for the combination of second-order centered dif- 
ferences and 1st, 2nd, 3rd, and 4th-order Runge-Kutta time-marching at a 
Courant number of unity. Also plot the phase speed error obtained using exact 
integration in time, i.e., that obtained using the spatial discretization alone. 
Note that the required c-roots for the various Runge-Kutta methods can be 
deduced from Eq. 6.69, without actually deriving the methods. Explain your 
results. 


Chapter 7 


STABILITY OF LINEAR 
SYSTEMS 


А general definition of stability is neither simple nor universal and depends on the 
particular phenomenon being considered. In the case of the nonlinear ODE’s of 
interest in fluid dynamics, stability is often discussed in terms of fixed points and 
attractors. In these terms a system is said to be stable in a certain domain if, from 
within that domain, some norm of its solution is always attracted to the same fixed 
point. These are important and interesting concepts, but we do not dwell on them in 
this work. Our basic concern is with time-dependent ODE's and OAE 's in which the 
coefficient matrices are independent of both u and t; see Section 4.2. We will refer 
to such matrices as stationary. Chapters 4 and 6 developed the representative forms 
of ODE’s generated from the basic PDE’s by the semidiscrete approach, and then 
the OAE’s generated from the representative ODE's by application of time-marching 
methods. These equations are represented by 


du a ru 

се = Ai - f) (7.1) 
and 

NUN S (7.3) 


respectively. For a one-step method, the latter form is obtained by applying a time- 
marching method to the generic ODE form in a fairly straightforward manner. For 
example, the explicit Euler method leads to C = I + hA, and ӯ, = f (nh). Methods 
involving two or more steps can always be written in the form of Eq. 7.2 by introducing 
new dependent variables. Note also that only methods in which the time and space 
discretizations are treated separately can be written in an intermediate semi-discrete 
form such as Eq. 7.1. The fully-discrete form, Eq. 7.2 and the associated stability 


definitions and analysis are applicable to all methods. 
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7.1 Dependence on the Eigensystem 


Our definitions of stability are based entirely on the behavior of the homogeneous 
parts of Eqs. 7.1 and 7.2. The stability of Eq. 7.1 depends entirely on the eigensys- 
tem! of A. The stability of Eq. 7.2 can often also be related to the eigensystem of 
its matrix. However, in this case the situation is not quite so simple since, in our 
applications to partial differential equations (especially hyperbolic ones), a stability 
definition can depend on both the time and space differencing. This is discussed in 
Section 7.4. Analysis of these eigensystems has the important added advantage that 
it gives an estimate of the rate at which a solution approaches a steady-state if a 
system is stable. Consideration will be given to matrices that have both complete 
and defective eigensystems, see Section 4.2.3, with a reminder that a complete system 
can be arbitrarily close to a defective one, in which case practical applications can 
make the properties of the latter appear to dominate. 


If A and C are stationary, we can, in theory at least, estimate their fundamental 
properties. For example, in Section 4.3.2 we found from our model ODE's for dif- 
fusion and periodic convection what could be expected for the eigenvalue spectrums 
of practical physical problems containing these phenomena, see Fig. 4.1. These ex- 
pectations are referred to many times in the following analysis of stability properties. 
They are important enough to be summarized by the following: 


e For diffusion dominated flows the A-eigenvalues tend to lie along the negative 
real axis. 


e For periodic convection-dominated flows the A-eigenvalues tend to lie along the 
imaginary axis. 


In many interesting cases, the eigenvalues of the matrices in Eqs. 7.1 and 7.2 
are sufficient to determine the stability. In previous chapters, we designated these 
eigenvalues as Àm and om for Eqs. 7.1 and 7.2, respectively, and we will find it 
convenient to examine the stability of various methods in both the complex А and 
complex o planes. 


l'This is not the case if the coefficient matrix depends on t even if it is linear. 
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7.2 Inherent Stability of ODE's 


7.2.1 The Criterion 


Here we state the standard stability criterion used for ordinary differential equa- 
tions. 


For a stationary matrix A, Eq. 7.1 is inherently stable if, when f is (7.3) 





constant, u remains bounded as t — oo. 


Note that inherent stability depends only on the transient solution of the ODE's. 


7.2.2 Complete Eigensystems 


If a matrix has a complete eigensystem, all of its eigenvectors are linearly independent, 
and the matrix can be diagonalized by a similarity transformation. In such a case it 
follows at once from Eq. 6.27, for example, that the ODE's are inherently stable if 
and only if 


R(Am) <0 for all m (7.4) 


This states that, for inherent stability, all of the А eigenvalues must lie on, or to 
the left of, the imaginary axis in the complex A plane. Inspecting Fig. 4.1, we see 
that this criterion is satisfied for the model ODE’s representing both diffusion and 
biconvection. It should be emphasized (as it is an important practical consideration 
in convection-dominated systems) that the special case for which А = +i is included 
in the domain of stability. In this case it is true that u does not decay as t > oo, but 
neither does it grow, so the above condition is met. Finally we note that for ODE’s 
with complete eigensystems the eigenvectors play no role in the inherent stability 
criterion. 


7.2.3 Defective Eigensystems 


In order to understand the stability of ODE's that have defective eigensystems, we 
inspect the nature of their solutions in eigenspace. For this we draw on the results in 
Sections 4.2.3 and especially on Eqs. 4.18 to 4.19 in that section. In an eigenspace 
related to defective systems the form of the representative equation changes from a 
single equation to a Jordan block. For example, instead of Eq. 4.45 a typical form of 
the homogeneous part might be 
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for which one finds the solution 
u(t) = u,(0)e* 
ш(@ = [us(0) + u (0) 
1 
ualt) (0) + о(0) + gr ex (7.5) 


Inspecting this solution, we see that for such cases condition 7.4 must be modified 
to the form 


R(Am) <0 for all m (7.6) 


since for pure imaginary А, из and из would grow without bound (linearly or quadrat- 
ically) if u2(0) Z 0 or u1(0) Z 0. Theoretically this condition is sufficient for stability 
in the sense of Statement 7.3 since t*e—!** — 0 as t — оо for all non-zero e. However, 
in practical applications the criterion may be worthless since there may be a very 
large growth of the polynomial before the exponential “takes over” and brings about 
the decay. Furthermore, on a computer such a growth might destroy the solution 
process before it could be terminated. 

Note that the stability condition 7.6 excludes the imaginary axis which tends to be 
occupied by the eigenvalues related to biconvection problems. However, condition 7.6 
is of little or no practical importance if significant amounts of dissipation are present. 


7.3 Numerical Stability of ОЛЕ ’s 


7.3.1 The Criterion 
The OAE companion to Statement 7.3 is 


For a stationary matrix C, Eq. 7.2 is numerically stable if, when g is (7.7) 





constant, un remains bounded as n — oo. 


We see that numerical stability depends only on the transient solution of the OAE 's. 
'This definition of stability is sometimes referred to as asymptotic or time stability. 

As we stated at the beginning of this chapter, stability definitions are not unique. 
A definition often used in CFD literature stems from the development of PDE so- 
lutions that do not necessarily follow the semidiscrete route. In such cases it is 
appropriate to consider simultaneously the effects of both the time and space ap- 
proximations. А time-space domain is fixed and stability is defined in terms of what 
happens to some norm of the solution within this domain as the mesh intervals go to 
zero at some constant ratio. We discuss this point of view in Section 7.4. 
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7.3.2 Complete Eigensystems 


Consider a set of ОЛЕ 's governed by a complete eigensystem. The stability criterion, 
according to the condition set in Eq. 7.7, follows at once from a study of Eq. 6.28 
and its companion for multiple o-roots, Eq. 6.36. Clearly, for such systems a time- 
marching method is numerically stable if and only if 


(ст), € 1 for all m and k (7.8) 


This condition states that, for numerical stability, all of the c eigenvalues (both 
principal and spurious, if there are any) must lie on or inside the unit circle in the 
complex o-plane. 

This definition of stability for OAE 's is consistent with the stability definition for 
ODE's. Again the sensitive case occurs for the periodic-convection model which places 
the “correct” location of the principal o—root precisely on the unit circle where the 
solution is only neutrally stable. Further, for a complete eigensystem, the eigenvectors 
play no role in the numerical stability assessment. 


7.3.3 Defective Eigensystems 


The discussion for these systems parallels the discussion for defective ODE’s. Examine 
Eq. 6.18 and note its similarity with Eq. 7.5. We see that for defective OAE's the 
required modification to 7.8 is 


(съ), « 1 for all m and k (7.9) 


since defective systems do not guarantee boundedness for |o| = 1, for example in Eq. 
7.5 if |c| = 1 and either u2(0) or ш. (0) Z 0 we get linear or quadratic growth. 


7.4 Time-Space Stability and Convergence of OAE's 


Let us now examine the concept of stability in a different way. In the previous 
discussion we considered in some detail the following approach: 


1. The PDE's are converted to ODE's by approximating the space derivatives on 
a finite mesh. 


2. Inherent stability of the ODE's is established by guaranteeing that (A) < 0. 


3. Time-march methods are developed which guarantee that |o(Ah)| € 1 and this 
is taken to be the condition for numerical stability. 
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This does guarantee that a stationary system, generated from a PDE on some fixed 
space mesh, will have a numerical solution that is bounded as t = nh > oo. This 
does not guarantee that desirable solutions are generated in the time march process 
as both the time and space mesh intervals approach zero. 

Now let us define stability in the time-space sense. First construct a finite time- 
space domain lying within 0 € z € L and 0 € t € T. Cover this domain with a grid 
that is equispaced in both time and space and fix the mesh ratio by the equation? 


_ At 
~ Ag 


Cn 


Next reduce our ОЛЕ approximation of the PDE to a two-level (i.e., two time-planes) 
formula in the form of Eq. 7.2. The homogeneous part of this formula is 


Eq. 7.10 is said to be stable if any bounded initial vector, 49, produces a bounded 
solution vector, ŭn, as the mesh shrinks to zero for a fixed c,. This is the classical 
definition of stability. It is often referred to as Lax or Lax-Richtmyer stability. Clearly 
as the mesh intervals go to zero, the number of time steps, N, must go to infinity in 
order to cover the entire fixed domain, so the criterion in 7.7 is a necessary condition 
for this stability criterion. 

The significance of this definition of stability arises through Lax’s Theorem, which 
states that, if a numerical method is stable (in the sense of Lax) and consistent then 
it is convergent. А method is consistent if it produces no error (in the Taylor series 
sense) in the limit as the mesh spacing and the time step go to zero (with с„ fixed, in 
the hyperbolic case). This is further discussed in Section 7.8. А method is convergent 
if it converges to the exact solution as the mesh spacing and time step go to zero in 
this manner.? Clearly, this is an important property. 

Applying simple recursion to Eq. 7.10, we find 


uU do 


and using vector and matrix p-norms (see Appendix A) and their inequality relations, 
we have 


П 1 = Стао | S 1C" [| 110 [ICT ||| (7.11) 


?This ratio is appropriate for hyperbolic problems; a different ratio may be needed for parabolic 
problems such as the model diffusion equation. 

3In the CFD literature, the word converge is used with two entirely different meanings. Here we 
refer to a numerical solution converging to the exact solution of the PDE. Later we will refer to the 
numerical solution converging to a steady state solution. 
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Since the initial data vector is bounded, the solution vector is bounded if 
ПСП & 1 (7.12) 


where ||C|| represents any p-norm of C. This is often used as a sufficient condition 
for stability. 


Now we need to relate the stability definitions given in Eqs. 7.8 and 7.9 with that 
given in Eq. 7.12. In Eqs. 7.8 and 7.9, stability is related to the spectral radius of 
C, ie., its eigenvalue of maximum magnitude. In Eq. 7.12, stability is related to a 
p-norm of C. It is clear that the criteria are the same when the spectral radius is a 
true p-norm. 


Two facts about the relation between spectral radii and matrix norms are well 
known: 


1. The spectral radius of a matrix is its Lə norm when the matrix is normal, i.e., 
it commutes with its transpose. 


2. The spectral radius is the lower bound of all norms. 


Furthermore, when C is normal, the second inequality in Eq. 7.11 becomes an equal- 
ity. In this case, Eq. 7.12 becomes both necessary and sufficient for stability. From 
these relations we draw two important conclusions about the numerical stability of 
methods used to solve PDE's. 


e The stability criteria in Eqs. 7.8 and 7.12 are identical for stationary systems 
when the governing matrix is normal. This includes symmetric, asymmetric, 
and circulant matrices. These criteria are both necessary and sufficient for 
methods that generate such matrices and depend solely upon the eigenvalues of 
the matrices. 


e If the spectral radius of any governing matrix is greater than one, the method 
is unstable by any criterion. Thus for general matrices, the spectral radius 
condition is necessary but not sufficient for stability. 


^Actually the necessary condition is that the spectral radius of C be less than or equal to 1 + 
O(At), but this distinction is not critical for our purposes here. 
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7.5 Numerical Stability Concepts in the Complex 
c-Plane 


7.5.1 o-Root Traces Relative to the Unit Circle 


Whether or not the semi-discrete approach was taken to find the differencing approx- 
imation of a set of PDE’s, the final difference equations can be represented by 


Un+1 = Cii, — On 


Furthermore if C has a complete? eigensystem, the solution to the homogeneous part 
can always be expressed as 


Un = 410123, ++ CmOn Im Et: d CMO MEM 


where the om are the eigenvalues of C. If the semi-discrete approach is used, we can 
find a relation between the c and the A eigenvalues. This serves as a very convenient 
guide as to where we might expect the o-roots to lie relative to the unit circle in the 
complex o-plane. For this reason we will proceed to trace the locus of the c-roots 
as a function of the parameter Ah for the equations modeling diffusion and periodic 
convection. 


Locus of the exact trace 


Figure 7.1 shows the exact trace of the o-root if it is generated by e^" representing 
either diffusion or biconvection. In both cases the e represents the starting value 
where h = 0 and o = 1. For the diffusion model, АЛ is real and negative. As the 
magnitude of АЛ increases, the trace representing the dissipation model heads towards 
the origin as Ah = —oo. On the other hand, for the biconvection model, АЛ = iwh 
is always imaginary. As the magnitude of wh increases, the trace representing the 
biconvection model travels around the circumference of the unit circle, which it never 
leaves. We must be careful in interpreting с when it is representing е". The fact 
that it /;es on the unit circle means only that the amplitude of the representation is 
correct, it tells us nothing of the phase error (see Eq. 6.41). The phase error relates 
to the position on the unit circle. 


>The subject of defective eigensystems has been addressed. From now on we will omit further 
discussion of this special case. 

$Or, if you like, the parameter h for fixed values of А equal to -1 and i for the diffusiion and 
biconvection cases, respectively. 
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Examples of some methods 


Now let us compare the exact o-root traces with some that are produced by actual 
time-marching methods. Table 7.1 shows the A-c relations for a variety of methods. 
Figures 7.2 and 7.3 illustrate the results produced by various methods when they are 
applied to the model ODE's for diffusion and periodic-convection, Eqs. 4.4 and 4.5. 
It is implied that the behavior shown is typical of what will happen if the methods 
are applied to diffusion- (or dissipation-) dominated or periodic convection-dominated 
problems as well as what does happen in the model cases. Most of the important 
possibilities are covered by the illustrations. 


Explicit Euler 
—2Aho —1z0 Leapfrog 
(1+ 2Ah)o + АҺ = 0 AB2 
(1+ Ah)? + BAho — АҺ = 0 AB3 
— Ah) -1=0 Implicit Euler 
— 4Ah) – (14+ AR) =0 Trapezoidal 
1— ЗАА) – 40 +1 = 0 2nd О Backward 
— gh) - (1+ $AR)e + BAR =0 AM3 
(1+ SAh + EA h?)o + S5Ah(14- АҺ) =0 ABM3 
— (1+ 24h)a? + 3Aho — АҺ =0 Gazdag 
c —1—Ah — 5А =0 RK2 
о —-1—Ah — ФА? — ФАЗ — TR = 0 RK4 
c?(1— АҺ) — $Aho — (1+ АҺ) =0 Milne 4th 
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Table 7.1. Some А — с Relations 


a. Explicit Euler Method 


Figure 7.2 shows results for the explicit Euler method. When used for dissipation- 
dominated cases it is stable for the range -2< Ah <0. (Usually the magnitude of А has 
to be estimated and often it is found by trial and error). When used for biconvection 
the o- trace falls outside the unit circle for all finite h, and the method has no range 
of stability in this case. 


b. Leapfrog Method 


This is a two-root method, since there are two o’s produced by every А. When 
applied to dissipation dominated problems we see from Fig. 7.2 that the principal 
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a) Diffusion b) Convection 


Figure 7.1: Exact traces of o-roots for model equations. 


root is stable for a range of Ah, but the spurious root is not. In fact, the spurious 
root starts on the unit circle and falls outside of it for all R(AR) < 0. However, for 
biconvection cases, when А is pure imaginary, the method is not only stable, but it 
also produces а c that falls precisely on the unit circle in the range 0 < wh < 1. 
As was pointed out above, this does not mean, that the method is without error. 
Although the figure shows that there is a range of wh in which the leapfrog method 
produces no error in amplitude, it says nothing about the error in phase. More is said 
about this in Chapter 8. 


c. Second-Order Adams-Bashforth Method 


This is also а two-root method but, unlike the leapfrog scheme, the spurious root 
starts at the origin, rather than on the unit circle, see Fig. 7.2. Therefore, there is 
a range of real negative Ah for which the method will be stable. The figure shows 
that the range ends when Ah « —1.0 since at that point the spurious root leaves the 
circle and |o2| becomes greater than one. The situation is quite different when the 
A-root is pure imaginary. In that case as wh increases away from zero the spurious 
root remains inside the circle and remains stable for a range of wh. However, the 
principal root falls outside the unit circle for all wh > 0, and for the biconvection 
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Figure 7.2: Traces of o-roots for various methods. 


model equation the method is unstable for all A. 


d. Trapezoidal Method 


The trapezoidal method is a very popular one for reasons that are partially illustrated 
in Fig. 7.3. Its o-roots fall on or inside the unit circle for both the dissipating and 
the periodic convecting case and, in fact, it is stable for all values of Ah for which 
à itself is inherently stable. Just like the leapfrog method it has the capability of 
producing only phase error for the periodic convecting case, but there is a major 
difference between the two since the trapezoidal method produces no amplitude error 
for any wh — not just a limited range between 0 € wh < 1. 
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e. Gazdag Method 


The Gazdag method was designed to produce low phase error. Since its characteristic 
polynomial for o is a cubic (Table 7.1, no. 10), it must have two spurious roots in 
addition to the principal one. These are shown in Fig. 7.3. In both the dissipation 
and biconvection cases, a spurious root limits the stability. For the dissipating case, 
a spurious root leaves the unit circle when AA < -i and for the biconvecting case, 
when wh > 2. Note that both spurious roots are located at the origin when A — 0. 


f,g. Second- and Fourth-Order Runge-Kutta Methods, RK2 and RK4 


Traces of the o-roots for the second- and fourth-order Runge-Kutta methods are 
shown in Fig. 7.3. The figures show that both methods are stable for a range of 
Ah when Ah is real and negative, but that the range of stability for RK4 is greater, 
going almost all the way to -2.8, whereas RK2 is limited to -2. On the other hand for 
biconvection RK2 is unstable for all wh, whereas RK4 remains inside the unit circle 
for 0 < wh < 24/2. One can show that the ВКА stability limit is about |Ah| < 2.8 for 
all complex AA for which (A) < 0. 


7.5.2 Stability for Small At 


It is interesting to pursue the question of stability when the time step size, h, is small 
so that accuracy of all the A-roots is of importance. Situations for which this is not 
the case are considered in Chapter 8. 


Mild instability 


All conventional time-marching methods produce a principal root that is very close 
to eò for small values of Ah. Therefore, on the basis of the principal root, the 
stability of a method that is required to resolve a transient solution over a relatively 
short time span may be a moot issue. Such cases are typified by the AB2 and RK2 
methods when they are applied to a biconvection problem. Figs. 7.2c and 7.3f show 
that for both methods the principal root falls outside the unit circle and is unstable 
for all wh. However, if the transient solution of interest can be resolved in a limited 
number of time steps that are small in the sense of the figure, the error caused by this 
instability may be relatively unimportant. If the root had fallen inside the circle the 
method would have been declared stable but an error of the same magnitude would 
have been committed, just in the opposite direction. For this reason the AB2 and the 
RK2 methods have both been used in serious quantitative studies involving periodic 
convection. This kind of instability is referred to as mild instability and is not a 
serious problem under the circumstances discussed. 
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Figure 7.3: Traces of o-roots for various methods (cont'd). 
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Catastrophic instability 


There is a much more serious stability problem for small h that can be brought about 
by the existence of certain types of spurious roots. One of the best illustrations of this 
kind of problem stems from a critical study of the most accurate, explicit, two-step, 
linear multistep method (see Table 7.1): 


Un41 = —4us + 5u, + 2h (2v, + x) (7.13) 


One can show, using the methods given in Section 6.6, that this method is third- 
order accurate both in terms of ету and er,, so from an accuracy point of view it is 
attractive. However, let us inspect its stability even for very small values of Ah. This 
can easily be accomplished by studying its characteristic polynomial when Ah — 0. 
From Eq. 7.13 it follows that for Ah = 0, P(E) = E? + AE — 5. Factoring P(o) = 0 
we find P(o) = (c — 1)(c + 5) = 0. There are two o-roots; с, the principal one, 
equal to 1, and o», a spurious one, equal to -5!! 

In order to evaluate the consequences of this result, one must understand how 
methods with spurious roots work in practice. We know that they are not self start- 
ing, and the special procedures chosen to start them initialize the coefficients of the 
spurious roots, the ск for k > 1 in Eq. 6.36. If the starting process is well designed 
these coefficients are forced to be very small, and if the method is stable, they get 
smaller with increasing n. However, if the magnitude of one of the spurious ø is 
equal to 5, one can see disaster is imminent because (5)? zz 107. Even a very small 
initial value of cmg is quickly overwhelmed. Such methods are called catastrophically 
unstable and are worthless for most, if not all, computations. 


Milne and Adams type methods 


If we inspect the c-root traces of the multiple root methods in Figs. 7.2 and 7.3, we 
find them to be of two types. One type is typified by the leapfrog method. In this 
case a spurious root falls on the unit circle when h — 0. The other type is exemplified 
by the 2nd-order Adams-Bashforth and Gazdag methods. In this case all spurious 
roots fall on the origin when h — 0. 

The former type is referred to as a Milne Method. Since at least one spurious root 
for a Milne method always starts on the unit circle, the method is likely to become 
unstable for some complex A as h proceeds away from zero. On the basis of a Taylor 
series expansion, however, these methods are generally the most accurate insofar as 
they minimize the coefficient in the leading term for er;. 

The latter type is referred to as an Adams Method. Since for these methods 
all spurious methods start at the origin for h = 0, they have a guaranteed range of 
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stability for small enough h. However, on the basis of the magnitude of the coefficient 
in the leading Taylor series error term, they suffer, relatively speaking, from accuracy. 

For a given amount of computational work, the order of accuracy of the two 
types is generally equivalent, and stability requirements in CFD applications generally 
override the (usually small) increase in accuracy provided by a coefficient with lower 
magnitude. 


7.6 Numerical Stability Concepts in the Complex 
Ah Plane 


7.6.1 Stability for Large Л. 


The reason to study stability for small values of h is fairly easy to comprehend. 
Presumably we are seeking to resolve some transient and, since the accuracy of the 
transient solution for all of our methods depends on the smallness of the time-step, 
we seek to make the size of this step as small as possible. On the other hand, the 
cost of the computation generally depends on the number of steps taken to compute 
a solution, and to minimize this we wish to make the step size as large as possible. 
In the compromise, stability can play a part. Aside from ruling out catastrophically 
unstable methods, however, the situation in which all of the transient terms are 
resolved constitutes a rather minor role in stability considerations. 

By far the most important aspect of numerical stability occurs under conditions 
when: 


e One has inherently stable, coupled systems with A-eigenvalues having widely 
separated magnitudes. 


or 


e We seek only to find a steady-state solution using a path that includes the 
unwanted transient. 


In both of these cases there exist in the eigensystems relatively large values of 
|Ah| associated with eigenvectors that we wish to drive through the solution process 
without any regard for their individual accuracy in eigenspace. This situation is 
the major motivation for the study of numerical stability. It leads to the subject of 
stiffness discussed in the next chapter. 
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7.6.2 Unconditional Stability, A-Stable Methods 


Inherent stability of a set of ODE’s was defined in Section 7.2 and, for coupled sets 
with a complete eigensystem, it amounted to the requirement that the real parts of all 
À eigenvalues must lie on, or to the left of, the imaginary axis in the complex A plane. 
This serves as an excellent reference frame to discuss and define the general stability 
features of time-marching methods. For example, we start with the definition: 


A numerical method is unconditionally stable if it is stable for all ОРЕ” 


that are inherently stable. 





A method with this property is said to be A-stable. A method is A,-stable if 
the region of stability contains the negative real axis in the complex Ah plane, and 
I-stable if it contains the entire imaginary axis. By applying a fairly simple test for 
A-stability in terms of positive real functions to the class of two-step LMM’s given in 
Section 6.59, one finds these methods to be A-stable if and only if 


1 
0 2 pr 2 (7.14) 
1 
— (7.15) 
1 
SS TQ (7.16) 


A set of A-stable implicit methods is shown in Table 7.2. 


Method 


$ 

0 Implicit Euler 

0 Trapezoidal 

1/2 2nd О Backward 

0 Adams type 
—1/2 -1/3 Lees 

—1/2 -1/2 Two-step trapezoidal 
-1/6 -2/9 A-contractive 


Table 7.2. Some unconditionally stable (A-stable) implicit methods. 





Notice that none of these methods has an accuracy higher than second-order. It can 
be proved that the order of an A-stable LMM cannot exceed two, and, furthermore 
that of all 2nd-order, A-stable methods the trapezoidal method has the smallest 
truncation error. 
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Returning to the stability test using positive real functions one can show that a 
two-step LMM is A,-stable if and only if 


ees Е (7.17) 
1 

ge (7.18) 

0<0—ф (7.19) 


For first-order accuracy, the inequalities 7.17 to 7.19 are less stringent than 7.14 to 
7.16. For second-order accuracy, however, the parameters (0, £, p) are related by the 
condition 


1 
Wu oues 


and the two sets of inequalities reduce to the same set which is 


€< 20-1 (7.20) 
pu -5 (7.21) 


Hence, two-step, second-order accurate LMM’s that are A-stable and A,-stable share 
the same (q,£,0) parameter space. Although the order of accuracy of an A-stable 
method cannot exceed two, A,-stable LMM methods exist which have an accuracy of 
arbitrarily high order. 

It has been shown that for a method to be I-stable it must also be A-stable. 
Therefore, no further discussion is necessary for the special case of I-stability. 

It is not difficult to prove that methods having a characteristic polynomial for 
which the coefficient of the highest order term in E is unity’ can never be uncondi- 
tionally stable. This includes all explicit methods and predictor-corrector methods 
made up of explicit sequences. Such methods are referred to, therefore, as condition- 
ally stable methods. 


7.6.3 Stability Contours in the Complex Ah Plane. 


A very convenient way to present the stability properties of a time-marching method 
is to plot the locus of the complex AA for which |o| = 1, such that the resulting 
contour goes through the point Ah = 0. Here |o| refers to the maximum absolute 
value of any c, principal or spurious, that is a root to the characteristic polynomial for 


"Or can be made equal to unity by a trivial normalization (division by a constant independent of 
Ah). The proof follows from the fact that the coefficients of such a polynomial are sums of various 
combinations of products of all its roots. 
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Figure 7.4: Stability Contours for the 0-method. 


a given Ah. It follows from Section 7.3 that on one side of this contour the numerical 
method is stable while on the other, it is unstable. We refer to it, therefore, as a 
stability contour. 

Typical stability contours for both explicit and implicit methods are illustrated in 
Fig. 7.4, which is derived from the one-root 0-method given in Section 6.5.4. 


Contours for explicit methods 


Fig. 7.4a shows the stability contour for the explicit Euler method. In the following 
two ways it is typical of all stability contours for explicit methods: 


1. The contour encloses a finite portion of the left-half complex Ah-plane. 
2. The region of stability is inside the boundary, and therefore, it is conditional. 


However, this method includes no part of the imaginary axis (except for the origin) 
and so it is unstable for the model biconvection problem. Although several explicit 
methods share this deficiency (e.g., AB2, RK2), several others do not (e.g., leapfrog, 
Gazdag, RK3, RK4), see Figs. 7.5 and 7.6. Notice in particular that the third- and 
fourth-order Runge-Kutta methods, Fig. 7.6, include a portion of the imaginary axis 
out to +1.9i and +2,/2i, respectively. 


Contours for unconditionally stable implicit methods 


Fig. 7.4c shows the stability contour for the implicit Euler method. It is typical of 
many stability contours for unconditionally stable implicit methods. Notice that the 
method is stable for the entire range of complex AA that fall outside the boundary. 
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Figure 7.5: Stability Contours for some explicit methods. 
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Figure 7.6: Stability Contours for Runge-Kutta methods. 
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Figure 7.7: Stability Contours for the 2 unconditionally stable implicit methods. 


This means that the method is numerically stable even when the ODE’s that it is 
being used to integrate are inherently unstable. Some other implicit unconditionally 
stable methods with the same property are shown in Fig. 7.7. In all of these cases 
the imaginary axis is part of the stable region. 

Not all unconditionally stable methods are stable in some regions where the ODE’s 
they are integrating are inherently unstable. The classic example of a method that 
is stable only when the generating ODE’s are themselves inherently stable is the 
trapezoidal method, i.e., the special case of the 0-method for which 0 = 5. The 
stability boundary for this case is shown in Fig. 7.4b. The boundary is the imaginary 
axis and the numerical method is stable for AA lying on or to the left of this axis. 
Two other methods that have this property are the two-step trapezoidal method 


/ / 
Un+1 = Un-1 + h(w, ys + v, 1) 


and a method due to Lees 


2 
Маф = Un—1 + sh (una +u, + tai) 


Notice that both of these methods are of the Milne type. 


Contours for conditionally stable implicit methods 


Just because a method is implicit does not mean that it is unconditionally stable. 
Two illustrations of this are shown in Fig. 7.8. One of these is the Adams-Moulton 
3rd-order method (no. 8, Table 7.1). Another is the 4th-order Milne method given 
by the point operator 


1 
Un41 = Un—1 + sh ua + 4ш, + шл] 


and shown in Table 7.1 as no. 13. It is stable only for А = +iw when 0 < w < V3. 
Its stability boundary is very similar to that for the leapfrog method (see Fig. 7.5b). 
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Figure 7.8: Stability Contours for the 2 conditionally stable implicit methods. 


7.7 Fourier Stability Analysis 


By far the most popular form of stability analysis for numerical schemes is the Fourier 
or von Neumann approach. ‘This analysis is usually carried out on point operators and 
it does not depend on an intermediate stage of ODE’s. Strictly speaking it applies 
only to difference approximations of PDE’s that produce OAE’s which are linear, 
have no space or time varying coefficients, and have periodic boundary conditions.® 
In practical application it is often used as a guide for estimating the worthiness of 
a method for more general problems. It serves as a fairly reliable necessary stability 
condition, but it is by no means a sufficient one. 


7.7.1 The Basic Procedure 


One takes data from a “typical” point in the flow field and uses this as constant 
throughout time and space according to the assumptions given above. Then one 
imposes a spatial harmonic as an initial value on the mesh and asks the question: 
Will its amplitude grow or decay in time? The answer is determined by finding the 
conditions under which 


ux f| = e” - om (7.22) 


is a solution to the difference equation, where к is real and &Az lies in the range 
0 € &Az < m. Since, for the general term, 


(п) _ paltHAt) | pik(z+mAz) ae 
j 


ы. е2! . gimAz 


u 


3Another way of viewing this is to consider it as an initial value problem on an infinite space 
domain. 
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the quantity ut 


expressions, we find the term e 


is common to every term and can be factored out. In the remaining 
аА which we represent by с, thus: 


о = е°^* 


. п . . 
Then, since e% = (eo4*) — g^, it is clear that 


For numerical stability |о| < 1 (7.23) 


and the problem is to solve for the o’s produced by any given method and, as a 
necessary condition for stability, make sure that, in the worst possible combination 
of parameters, condition 7.23 is satisfied?. 


7.7.2 Some Examples 


The procedure can best be explained by examples. Consider as a first example the 
finite difference approximation to the model diffusion equation known as Richardson’s 
method of overlapping steps. This was mentioned in Section 4.1 and given as Eq. 
4.1: 


n41 т—1 т т т 


Substitution of Eq. 7.22 into Eq. 7.24 gives the relation 


2AL, , : 
— —1 їкАхж __ —1кАх 
с = + тл (е 2+е ) 
or 
AvAt 
a — cos rAc)| с-1= 0 (7.25) 
——MÁ— 

2b 


Thus Eq. 7.22 is a solution of Eq. 7.24 if ø is a root of Eq. 7.25. The two roots of 


7.25 are 
01,2 = —b+ vb? +1 


from which it is clear that one |o| is always > 1. We find, therefore, that by the 
Fourier stability test, Richardson’s method of overlapping steps is unstable for all v, 
к and At. 


°If boundedness is required in a finite time domain, the condition is often presented as |o| < 
14+ O(At). 
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As another example consider the finite-difference approximation for the model 
biconvection equation 


(ntt) (ш) | QAM (а) (m) 
и uj Gag) (7.26) 


In this case 


ад? 
o = 1 ——-i-sinkAx 
Ат 
from which it is clear that |o| > 1 for all nonzero а and к. Thus we have another 
finite-difference approximation that, by the Fourier stability test, is unstable for any 
choice of the free parameters. 


7.7.3 Relation to Circulant Matrices 


The underlying assumption in a Fourier stability analysis is that the C matrix, deter- 
mined when the differencing scheme is put in the form of Eq. 7.2, is circulant. Such 
being the case, the е in Eq. 7.22 represents an eigenvector of the system, and the 
two examples just presented outline a simple procedure for finding the eigenvalues of 
the circulant matrices formed by application of the two methods to the model prob- 
lems. The choice of o for the stability parameter in the Fourier analysis, therefore, 
is not an accident. It is exactly the same с we have been using in all of our previous 
discussions, but arrived at from a different perspective. 

If we examine the preceding examples from the viewpoint of circulant matrices 
and the semi-discrete approach, the results present rather obvious conclusions. ‘The 
space differencing in Richardson’s method produces the matrix s- В, (1, —2, 1) where 
s is a positive scalar coefficient. From Appendix B we find that the eigenvalues of this 
matrix are real negative numbers. Clearly, the time-marching is being carried out by 
the leapfrog method and, from Fig. 7.5, this method is unstable for all eigenvalues with 
negative real parts. On the other hand, the space matrix in Eq. 7.26 is B,(— 1,0, 1), 
and according to Appendix B, this matrix has pure imaginary eigenvalues. However, 
in this case the explicit Euler method is being used for the time-march and, according 
to Fig. 7.4, this method is always unstable for such conditions. 


7.8 Consistency 


Consider the model equation for diffusion analysis 


E es ee (7.27) 
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Many years before computers became available (1910, in fact), Lewis F. Richardson 
proposed a method for integrating equations of this type. We presented his method 
in Eq. 4.2 and analyzed its stability by the Fourier method in Section 7.7. 

In Richardson's time, the concept of numerical instability was not known. How- 
ever, the concept is quite clear today and we now know immediately that his approach 
would be unstable. As a semi-discrete method it can be expressed in matrix notation 
as the system of ODE's: 


PE da 


a = AGB 72, 1)u + (be) (7.28) 


with the leapfrog method used for the time march. Our analysis in this Chapter 
revealed that this is numerically unstable since the A-roots of B(1,-2,1) are all real 
and negative and the spurious g-root in the leapfrog method is unstable for all such 
cases, see Fig. 7.5b. 

The method was used by Richardson for weather prediction, and this fact can 
now be a source of some levity. In all probability, however, the hand calculations (the 
only approach available at the time) were not carried far enough to exhibit strange 
phenomena. We could, of course, use the 2nd-order Runge-Kutta method to integrate 
Eq. 7.28 since it is stable for real negative A's. It is, however, conditionally stable 
and for this case we are rather severely limited in time step size by the requirement 
At Аз? /(2ь). 

There are many ways to manipulate the numerical stability of algorithms. One of 
them is to introduce mixed time and space differencing, a possibility we have not yet 
considered. For example, we introduced the DuFort-Frankel method in Chapter 4: 

(n+1) (n—1) 
шщ") = yD 4 a ul), — (s FA | ru 





E 2 








in which the central term in the space derivative in Eq. 4.2 has been replaced by its 
average value at two different time levels. Now let 


2vAt 
-SAVAI 7.29 
Аз? = 
and rearrange terms 
(1+ oy t? eu оў 7? + ош"), + ut 


There is no obvious ODE between the basic PDE and this final ОЛЕ. Hence, there 
is no intermediate A-root structure to inspect. Instead one proceeds immediately to 
the o-roots. 
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The simplest way to carry this out is by means of the Fourier stability analysis 
introduced in Section 7.7. This leads at once to 


(1 +a)o = (1 — о)о7 + а(е'^^= + poe) 
Or 
(1 +a)? — 2ac cos(kAx) — (1 — a) = 0 
The solution of the quadratic is 
acos KAT + V1 — o? sin? кЛт 
o = —— 
l+a 


There are 2M o-roots all of which are < 1 for any real o in the range 0 € a < оо. 
This means that the method is unconditionally stable! 

The above result seems too good to be true, since we have found an unconditionally 
stable method using an explicit combination of Lagrangian interpolation polynomials. 


The price we have paid for this is the loss of con- 
sistency with the original PDE. 


To prove this, we expand the terms in Eq. 7.29 in a Taylor series and reconstruct 
the partial differential equation we are actually solving as the mesh size becomes very 
small. For the time derivative we have 


1 (n+1) (n-1) niu D ds (n) 
Ar^ =U; | = (8и) + gat (ди) + 


and for the mixed time and space differences 


n n-1 n—1 n 
Ий «ый ыр Эз M 
Ах? Е (дм), Ax (дии); T: 
l 2 (n) 
pA (Ornax) 5 == 
1 Ату? » 
GA (RE) (Önnu)! haee (7.30) 


Replace the terms in Eq. 7.29 with the above expansions and take the limit as 
At, Ат — 0. We find 
ðu Pu 5070 


ди _ u „дч 7.31 
о ае oy) 


where 
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Eq. 7.27 is parabolic. Eq. 7.31 is hyperbolic. Thus if At + 0 and Az — 0 in such 
a way that AL remains constant, the equation we actually solve by the method in 
Eq. 7.29 is a wave equation, not a diffusion equation. In such a case Eq. 7.29 is not 
uniformly consistent with the equation we set out to solve even for vanishingly small 
step sizes. The situation is summarized in Table 7.3. If At — 0 and Az — 0 in such 
a way that 25 remains constant, then r? is O(At), and the method is consistent. 
However, this leads to a constraint on the time step which is just as severe as that 
imposed by the stability condition of the second-order Runge-Kutta method shown 
in the table. 


2nd-order Runge-Kutta DuFort-Frankel 


For Stability 


At < oo 


Conditionally Stable Unconditionally Stable 
For Consistency 
Uniformly Consistent Conditionally Consistent, 
. ди __ 82 
Approximates $7 = 1/9235 
only if 


v( «e 
Therefore 


At < Ату 


Table 7.3: Summary of accuracy and consistency conditions for RK2 and 
Du Fort-Frankel methods. є = an arbitrary error bound. 





7.9 Problems 


1. Consider the ODE 
du 
ГА 
= — >= A 
u a ut+f 


with 





10: <a exp =j 
A= p =i 1| , f=] 0 
10 l3 0 
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(a) Find the eigenvalues of A using a numerical package. What is the steady- 
state solution? How does the ODE solution behave in time? 


(b) Write a code to integrate from the initial condition u(0) = [1, 1, 1|" from 
time t = 0 using the explicit Euler, implicit Euler, and MacCormack meth- 
ods. In all three cases, use h = 0.1 for 1000 time steps, h = 0.2 for 500 
time steps, Л = 0.4 for 250 time steps and h = 1.0 for 100 time steps. 
Compare the computed solution with the exact steady solution. 


(c) Using the A-o relations for these three methods, what are the expected 
bounds on A for stability? Are your results consistent with these bounds? 


2. (a) Compute a table of the numerical values of the o-roots of the 2nd-order 
Adams-Bashforth method when A — i. Take h in intervals of 0.05 from 0 
to 0.80 and compute the absolute values of the roots to at least 6 places. 


(b) Plot the trace of the roots in the complex o-plane and draw the unit circle 
on the same plot. 


(c) Repeat the above for the RK2 method. 


3. When applied to the linear convection equation, the widely known Lax-Wendroff 
method gives: 


uj =иў— gea = №1) + Спб йы + uj i) 


where Cn, known as the Courant (or CFL) number, is ah/Az. Using Fourier 
stability analysis, find the range of С„ for which the method is stable. 


4. Determine and plot the stability contours for the Adams-Bashforth methods of 
order 1 through 4. Compare with the Runge-Kutta methods of order 1 through 
4. 


5. Recall the compact centered 3-point approximation for a first derivative: 


3 
(05)j-1 + 4(8zu)5 + (дәш) = Kz ga — шу-л) 


By replacing the spatial index 7 by the temporal index n, obtain a time-marching 
method using this formula. What order is the method? Is it explicit or implicit? 
Is it a two-step LMM? If so, to what values of £, 0, and $ (in Eq. 6.59) does it 
correspond? Derive the A-o relation for the method. Is it A-stable, A,-stable, 
or I-stable? 
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. Write the ODE system obtained by applying the 2nd-order centered difference 


approximation to the spatial derivative in the model diffusion equation with 
periodic boundary conditions. Using Appendix B.4, find the eigenvalues of the 
spatial operator matrix. Given that the A-o relation for the 2nd-order Adams- 
Bashforth method is 


о? — (14-3Ah/2)a + Ah/2 = 0 


show that the maximum stable value of |AA| for real negative A, i.e., the point 
where the stability contour intersects the negative real axis, is obtained with 
Ah — —1. Using the eigenvalues of the spatial operator matrix, find the maxi- 
mum stable time step for the combination of 2nd-order centered differences and 
the 2nd-order Adams-Bashforth method applied to the model diffusion equa- 
tion. Repeat using Fourier analysis. 


. Consider the following PDE: 


ди du 
дї ~ 9g? 


where i = /—1. Which explicit time-marching methods would be suitable for 
integrating this PDE, if 2nd-order centered differencing is used for the spa- 
tial differences and the boundary conditions are periodic? Find the stability 
condition for one of these methods. 


. Using Fourier analysis, analyze the stability of first-order backward differenc- 


ing coupled with explicit Euler time marching applied to the linear convection 
equation with positive a. Find the maximum Courant number for stability. 


. Consider the linear convection with a positive wave speed as in problem 8. Apply 


a Dirichlet boundary condition at the left boundary. No boundary condition is 
permitted at the right boundary. Write the system of ODE's which results from 
first-order backward spatial differencing in matrix-vector form. Using Appendix 
B.1, find the A-eigenvalues. Write the OAE which results from the application 
of explicit Euler time marching in matrix-vector form, i.e., 


Un+1 = Cùn — On 


Write C in banded matrix notation and give the entries of g. Using the A- 
с relation for the explicit Euler method, find the o-eigenvalues. Based on 
these, what is the maximum Courant number allowed for asymptotic stability? 
Explain why this differs from the answer to problem 8. Hint: is C normal? 


Chapter 8 


CHOICE OF TIME-MARCHING 
METHODS 


In this chapter we discuss considerations involved in selecting a time-marching method 
for a specific application. Examples are given showing how time-marching methods 
can be compared in a given context. Ап important concept underlying much of this 
discussion is stiffness, which is defined in the next section. 


8.1 Stiffness Definition for ODE’s 


8.1.1 Relation to A-Eigenvalues 


The introduction of the concept referred to as “stiffness” comes about from the nu- 
merical analysis of mathematical models constructed to simulate dynamic phenom- 
ena containing widely different time scales. Definitions given in the literature are 
not unique, but fortunately we now have the background material to construct a 
definition which is entirely sufficient for our purposes. 

We start with the assumption that our CFD problem is modeled with sufficient 
accuracy by a coupled set of ODE's producing an A matrix typified by Eq. 7.1. 
Any definition of stiffness requires a coupled system with at least two eigenvalues, 
and the decision to use some numerical time-marching or iterative method to solve 
it. The difference between the dynamic scales in physical space is represented by 
the difference in the magnitude of the eigenvalues in eigenspace. In the following 
discussion we concentrate on the transient part of the solution. The forcing function 
may also be time varying in which case it would also have a time scale. However, 
we assume that this scale would be adequately resolved by the chosen time-marching 
method, and, since this part of the ODE has no effect on the numerical stability of 
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Figure 8.1: Stable and accurate regions for the explicit Euler method. 


the homogeneous part, we exclude the forcing function from further discussion in this 
section. 


Consider now the form of the exact solution of a system of ODE’s with a com- 
plete eigensystem. This is given by Eq. 6.27 and its solution using a one-root, time- 
marching method is represented by Eq. 6.28. For a given time step, the time integra- 
tion is an approximation in eigenspace that is different for every eigenvector 2. In 
many numerical applications the eigenvectors associated with the small |А„,| are well 
resolved and those associated with the large |Am| are resolved much less accurately, 
if at all. The situation is represented in the complex Ah plane in Fig. 8.1. In this 
figure the time step has been chosen so that time accuracy is given to the eigenvectors 
associated with the eigenvalues lying in the small circle and stability without time 
accuracy is given to those associated with the eigenvalues lying outside of the small 
circle but still inside the large circle. 


The whole concept of stiffness in CFD arises from the fact that we often do 
not need the time resolution of eigenvectors associated with the large |А. | in 


the transient solution, although these eigenvectors must remain coupled into 
the system to maintain a high accuracy of the spatial resolution. 
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8.1.2 Driving and Parasitic Eigenvalues 


For the above reason it is convenient to subdivide the transient solution into two 
parts. First we order the eigenvalues by their magnitudes, thus 





I S Dal S++ € [Au] (8.1) 
Then we write 
Transient P Amt 2 M TE 
Solution ~ 2. Сте Хт its Сте" 2 (8.2) 
Driving Parasitic 


This concept is crucial to our discussion. Rephrased, it states that we can separate 
our eigenvalue spectrum into two groups; опе [Ау — à] called the driving eigenvalues 
(our choice of a time-step and marching method must accurately approximate the time 
variation of the eigenvectors associated with these), and the other, [Ат — Ay], called 
the parasitic eigenvalues (no time accuracy whatsoever is required for the eigenvectors 
associated with these, but their presence must not contaminate the accuracy of the 
complete solution). Unfortunately, we find that, although time accuracy requirements 
are dictated by the driving eigenvalues, numerical stability requirements are dictated 
by the parasitic ones. 


8.1.3 Stiffness Classifications 


The following definitions are somewhat useful. An inherently stable set of ODE’s is 
stiff if 
АЫ < [Ам 
In particular we define the ratio 
C, = |Ам| / Il 
and form the categories 
Mildly-stiff C, «10 
Strongly-stiff LOPS C, < 10° 
Extremely-stiff 10 < С, < 108 
Pathologically-stiff 10? < С, 


It should be mentioned that the gaps in the stiff category definitions are intentional 
because the bounds are arbitrary. It is important to notice that these definitions 
make no distinction between real, complex, and imaginary eigenvalues. 
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8.2 Relation of Stiffness to Space Mesh Size 


Many flow fields are characterized by a few regions having high spatial gradients of 
the dependent variables and other domains having relatively low gradient phenomena. 
As a result it is quite common to cluster mesh points in certain regions of space and 
spread them out otherwise. Examples of where this clustering might occur are at a 
shock wave, near an airfoil leading or trailing edge, and in a boundary layer. 

One quickly finds that this grid clustering can strongly affect the eigensystem of 
the resulting A matrix. In order to demonstrate this, let us examine the eigensystems 
of the model problems given in Section 4.3.2. The simplest example to discuss relates 
to the model diffusion equation. In this case the eigenvalues are all real, negative 
numbers that automatically obey the ordering given in Eq. 8.1. Consider the case 
when all of the eigenvalues are parasitic, i.e., we are interested only in the converged 
steady-state solution. Under these conditions, the stiffness is determined by the ratio 
Am/Ai. A simple calculation shows that 


M AC NE NE ы (0) (2°) =- 
=A Vaated} ^ NAsS/X2/ — 





A Av 2 (5) Av 
д ———, siní |- | = -— 
М Ax? 2 Ат? 
and the ratio is E 
4 М+1 
Е й ( ) 
m/^ Aa? T 


The most important information found from this example is the fact that the 
stiffness of the transient solution is directly related to the grid spacing. Furthermore, 
in diffusion problems this stiffness is proportional to the reciprocal of the space mesh 
size squared. For a mesh size M — 40, this ratio is about 680. Even for a mesh of 
this moderate size the problem is already approaching the category of strongly stiff. 

For the biconvection model a similar analysis shows that 


1 
[Aml ЛА © 5 
Here the stiffness parameter is still space-mesh dependent, but much less so than for 
diffusion-dominated problems. 

We see that in both cases we are faced with the rather annoying fact that the more 
we try to increase the resolution of our spatial gradients, the stiffer our equations tend 
to become. Typical CFD problems without chemistry vary between the mildly and 
strongly stiff categories, and are greatly affected by the resolution of a boundary layer 
since it is a diffusion process. Our brief analysis has been limited to equispaced 
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problems, but in general the stiffness of CFD problems is proportional to the mesh 
intervals in the manner shown above where the critical interval is the smallest one in 
the physical domain. 


8.3 Practical Considerations for Comparing Meth- 
ods 


We have presented relatively simple and reliable measures of stability and both the 
local and global accuracy of time-marching methods. Since there are an endless 
number of these methods to choose from, one can wonder how this information is 
to be used to pick a “best” choice for a particular problem. There is no unique 
answer to such a question. For example, it is, among other things, highly dependent 
upon the speed, capacity, and architecture of the available computer, and technology 
influencing this is undergoing rapid and dramatic changes as this is being written. 
Nevertheless, if certain ground rules are agreed upon, relevant conclusions can be 
reached. Let us now examine some ground rules that might be appropriate. It should 
then be clear how the analysis can be extended to other cases. 

Let us consider the problem of measuring the efficiency of a time-marching method 
for computing, over a fixed interval of time, an accurate transient solution of a coupled 
set of ODE's. The length of the time interval, Т, and the accuracy required of the 
solution are dictated by the physics of the particular problem involved. For example, 
in calculating the amount of turbulence in a homogeneous flow, the time interval 
would be that required to extract a reliable statistical sample, and the accuracy 
would be related to how much the energy of certain harmonics would be permitted 
to distort from a given level. Such a computation we refer to as an event. 

'The appropriate error measures to be used in comparing methods for calculating 
an event are the global ones, Era, Er, and Er,, discussed in Section 6.6.5, rather 
than the local ones ery, erg, and er, discussed earlier. 

'The actual form of the coupled ODE's that are produced by the semi-discrete 

approach is 
~ = F(u,t) 
At every time step we must evaluate the function F(u, t) at least once. This function 
is usually nonlinear, and its computation usually consumes the major portion of the 
computer time required to make the simulation. We refer to a single calculation of the 
vector F(u, t) as a function evaluation and denote the total number of such evaluations 
by Fev- 
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8.4 Comparing the Efficiency of Explicit Methods 


8.4.1 Imposed Constraints 


As mentioned above, the efficiency of methods can be compared only if one accepts a 
set of limiting constraints within which the comparisons are carried out. The follow 
assumptions bound the considerations made in this Section: 


1. The time-march method is explicit. 


2. Implications of computer storage capacity and access time are ignored. In some 
contexts, this can be an important consideration. 


3. The calculation is to be time-accurate, must simulate an entire event which 
takes a total time T', and must use a constant time step size, h, so that 


T = Nh 


where N is the total number of time steps. 


8.4.2 An Example Involving Diffusion 


Let the event be the numerical solution of 


du 

e жо 8.3 
from t — 0 to T — —1n(0.25) with u(0) — 1. Eq. 8.3 is obtained from our representa- 
tive ODE with А = —1, a = 0. Since the exact solution is u(t) = u(0)e 5, this makes 
the exact value of u at the end of the event equal to 0.25, i.e., u(T) = 0.25. To the 
constraints imposed above, let us set the additional requirement 


e The error in u at the end of the event, i.e., the global error, must be « 0.596. 


We judge the most efficient method as the one that satisfies these conditions and 
has the fewest number of evaluations, Fey. Three methods are compared — explicit 
Euler, AB2, and ККА. 

First of all, the allowable error constraint means that the global error in the am- 
plitude, see Eq. 6.48, must have the property: 


Ет, 


—7| < 0.005 
eT 
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Then, since h = T/N = —1n(0.25)/N, it follows that 
|1 — (ex (1n(:25)/N))" /.25| < 0.005 


where c is found from the characteristic polynomials given in Table 7.1. The results 
shown in Table 8.1 were computed using a simple iterative procedure. 


Method N h 01 Fu Ery 
Euler 193 .00718 .99282 193 .001248 worst 
AB2 16 .0866 .9172 16  .001137 
RK4 2 .6931 .5012 8  .001195 best 


Table 8.1: Comparison of time-marching methods for a simple dissipation problem. 





In this example we see that, for a given global accuracy, the method with the 
highest local accuracy is the most efficient on the basis of the expense in evaluating 
Fæ. Thus the second-order Adams-Bashforth method is much better than the first- 
order Euler method, and the fourth-order Runge-Kutta method is the best of all. The 
main purpose of this exercise is to show the (usually) great superiority of second-order 
over first-order time-marching methods. 


8.4.3 An Example Involving Periodic Convection 


Let us use as a basis for this example the study of homogeneous turbulence simulated 
by the numerical solution of the incompressible Navier-Stokes equations inside a cube 
with periodic boundary conditions on all sides. In this numerical experiment the 
function evaluations contribute overwhelmingly to the CPU time, and the number of 
these evaluations must be kept to an absolute minimum because of the magnitude of 
the problem. On the other hand, a complete event must be established in order to 
obtain meaningful statistical samples which are the essence of the solution. In this 
case, in addition to the constraints given in Section 8.4.1, we add the following: 


e The number of evaluations of F'(ii,t) is fixed. 


Under these conditions a method is judged as best when it has the highest global 
accuracy for resolving eigenvectors with imaginary eigenvalues. The above constraint 
has led to the invention of schemes that omit the function evaluation in the cor- 
rector step of a predictor-corrector combination, leading to the so-called incomplete 
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predictor-corrector methods. The presumption is, of course, that more efficient meth- 
ods will result from the omission of the second function evaluation. An example is 
the method of Gazdag, given in Section 6.8. Basically this is composed of an AB2 
predictor and a trapezoidal corrector. However, the derivative of the fundamental 
family is never found so there is only one evaluation required to complete each cycle. 
The A-o relation for the method is shown as entry 10 in Table 7.1. 

In order to discuss our comparisions we introduce the following definitions: 


e Let a k-evaluation method be defined as one that requires k evaluations of 
F (ii, t) to advance one step using that method’s time interval, h. 


e Let К represent the total number of allowable Fey. 


e Let hı be the time interval advanced in one step of a one-evaluation method. 


The Gazdag, leapfrog, and AB2 schemes are all 1-evaluation methods. The second 
and fourth order RK methods are 2- and 4-evaluation methods, respectively. For a 1- 
evaluation method the total number of time steps, N, and the number of evaluations, 
К, are the same, one evaluation being used for each step, so that for these methods 
h = hy. For a 2-evaluation method № = K/2 since two evaluations are used for 
each step. However, in this case, in order to arrive at the same time T after К 
evaluations, the time step must be twice that of a one-evaluation method so h = 2h. 
For a 4-evaluation method the time interval must be h = 4h,, etc. Notice that 
as k increases, the time span required for one application of the method increases. 
However, notice also that as k increases, the power to which сү is raised to arrive 
at the final destination decreases; see the Figure below. This is the key to the true 
comparison of time-march methods for this type of problem. 


Step sizes and powers of o for k-evaluation methods used to get to the same value 
of T if 8 evaluations are allowed. 





In general, after K evaluations, the global amplitude and phase error for k- 
evaluation methods applied to systems with pure imaginary A-roots can be written! 


Er, = 1- |os(ikohi)| * (8.4) 
1See Eqs. 6.38 and 6.39. 
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LÍ [oi (ikw hı Jlimaginäry 


K 
Er, = ШТ — — tan : 
k [os (tkwhy)|real 


(8.5) 


Consider a convection-dominated event for which the function evaluation is very 
time consuming. We idealize to the case where А = iw and set ш equal to one. The 
event must proceed to the time t — T — 10. We consider two maximum evaluation 
limits K = 50 and K = 100 and choose from four possible methods, leapfrog, AB2, 
Gazdag, and RK4. The first three of these are one-evaluation methods and the last 
one is a four-evaluation method. It is not difficult to show that on the basis of local 
error (made in a single step) the Gazdag method is superior to the RK4 method in 
both amplitude and phase. For example, for wh = 0.2 the Gazdag method produces 
а |c1| = 0.9992276 whereas for wh = 0.8 (which must be used to keep the number of 
evaluations the same) ће RK4 method produces a |с; | = 0.998324. However, we are 
making our comparisons on the basis of global error for a fixed number of evaluations. 

First of all we see that for a one-evaluation method hı = T/K. Using this, and the 
fact that w — 1, we find, by some rather simple calculations? made using Eqs. 8.4 and 
8.5, the results shown in Table 8.2. Notice that to find global error the Gazdag root 
must be raised to the power of 50 while the RK4 root is raised only to the power of 
50/4. On the basis of global error the Gazdag method is not superior to RK4 in either 
amplitude or phase, although, in terms of phase error (for which it was designed) it 
is superior to the other two methods shown. 


К leapfrog AB2 Gazdag RK4 
100 1.0 1.003 .995 .999 
50 1.0 1.022 .962 .979 


a. Amplitude, exact = 1.0. 


К leapfrog АВ2 Gazdag RK4 
100 —.96  —24 .45 .12 
50 -3.8 —9.8 1.5 1.5 


b. Phase error in degrees. 


Table 8.2: Comparison of global amplitude and phase errors for four methods. 





?'The cı root for the Gazdag method can be found using a numerical root finding routine to trace 
the three roots in the c-plane, see Fig. 7.3e. 
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Using analysis such as this (and also considering the stability boundaries) the 
RK4 method is recommended as a basic first choice for any explicit time-accurate 
calculation of a convection-dominated problem. 


8.5 Coping With Stiffness 


8.5.1 Explicit Methods 


The ability of a numerical method to cope with stiffness can be illustrated quite nicely 
in the complex Ah plane. А good example of the concept is produced by studying 
the Euler method applied to the representative equation. The transient solution is 
Un = (1 + АҺ)" and the trace of the complex value of Ah which makes |1 + Ah| = 1 
gives the whole story. In this case the trace forms a circle of unit radius centered at 
(—1,0) as shown in Fig. 8.1. If h is chosen so that all Ah in the ODE eigensystem 
fall inside this circle the integration will be numerically stable. Also shown by the 
small circle centered at the origin is the region of Taylor series accuracy. If some Ah 
fall outside the small circle but stay within the stable region, these AA are stiff, but 
stable. We have defined these Ал as parasitic eigenvalues. Stability boundaries for 
some explicit methods are shown in Figs. 7.5 and 7.6. 

For a specific example, consider the mildly stiff system composed of a coupled 
two-equation set having the two eigenvalues Ау = —100 and A» = —1. If uncoupled 
and evaluated in wave space, the time histories of the two solutions would appear as a 
rapidly decaying function in one case, and a relatively slowly decaying function in the 
other. Analytical evaluation of the time histories poses no problem since e71% quickly 
becomes very small and can be neglected in the expressions when time becomes large. 
Numerical evaluation is altogether different. Numerical solutions, of course, depend 
upon [c (A4, h)]" and no |o,,| can exceed one for any Àm in the coupled system or else 
the process is numerically unstable. 

Let us choose the simple explicit Euler method for the time march. The coupled 
equations in real space are represented by 


ui(n) = c(l S 100A)" i1 + со(1 = h)" zi» + (PS); 
u»(n) LI c(l = 100A)" x21 + со(1 = h)” x29 -F (PS); (8.6) 





We will assume that our accuracy requirements are such that sufficient accuracy is 
obtained as long as |Ah| < 0.1. This defines a time step limit based on accuracy 
considerations of h = 0.001 for A; and h = 0.1 for А. The time step limit based 
on stability, which is determined from Ау, is ^ = 0.02. We will also assume that 
€; = с = 1 and that an amplitude less than 0.001 is negligible. We first run 66 
time steps with Л = 0.001 in order to resolve the А; term. With this time step the 
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А term is resolved exceedingly well. After 66 steps, the amplitude of the A, term 
(1.е., (1 — 100A)”) is less than 0.001 and that of the A» term (i.e., (1 — h)") is 0.9361. 
Hence the A; term can now be considered negligible. To drive the (1 — h)” term to 
zero (i.e., below 0.001), we would like to change the step size to h = 0.1 and continue. 
We would then have a well resolved answer to the problem throughout the entire 
relevant time interval. However, this is not possible because of the coupled presence 
of (1 — 100A)", which in just 10 steps at h = 0.1 amplifies those terms by ~ 10°, far 
outweighing the initial decrease obtained with the smaller time step. In fact, with 
h — 0.02, the maximum step size that can be taken in order to maintain stability, 
about 339 time steps have to be computed in order to drive e^! to below 0.001. Thus 
the total simulation requires 405 time steps. 


8.5.2 Implicit Methods 


Now let us re-examine the problem that produced Eq. 8.6 but this time using an 
unconditionally stable implicit method for the time march. We choose the trapezoidal 
method. Its behavior in the AA plane is shown in Fig. 7.4b. Since this is also a one- 
root method, we simply replace the Euler с with the trapezoidal one and analyze the 
result. It follows that the final numerical solution to the ODE is now represented in 
real space by 


1 — 50h" 1 — 0.55" 
ш (п) = a (ar) 211 ra ) L12 T (PS); 








1+ 50h 1+ 0.5h 
1—50h\" 1—0.5h\" 
u»(n) = € (=) 291 T Сә inc) 299 T (PS); (8.7) 


In order to resolve the initial transient of the term e~!°", we need to use a step size 


of about h = 0.001. This is the same step size used in applying the explicit Euler 
method because here accuracy is the only consideration and a very small step size 
must be chosen to get the desired resolution. (It is true that for the same accuracy 
we could in this case use a larger step size because this is a second-order method, 
but that is not the point of this exercise). After 70 time steps the Ау term has 
amplitude less than 0.001 and can be neglected. Now with the implicit method we 
can proceed to calculate the remaining part of the event using our desired step size 
h = 0.1 without any problem of instability, with 69 steps required to reduce the 
amplitude of the second term to below 0.001. In both intervals the desired solution 
is second-order accurate and well resolved. It is true that in the final 69 steps one 
c-root is [1 — 50(0.1)]/[1 + 50(0.1)] = 0.666 ---, and this has no physical meaning 
whatsoever. However, its influence on the coupled solution is negligible at the end 
of the first 70 steps, and, since (0.666 ---)” < 1, its influence in the remaining 70 
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steps is even less. Actually, although this root is one of the principal roots in the 
system, its behavior for t > 0.07 is identical to that of a stable spurious root. The 
total simulation requires 139 time steps. 


8.5.3 A Perspective 


It is important to retain a proper perspective on a problem represented by the above 
example. It is clear that an unconditionally stable method can always be called upon 
to solve stiff problems with a minimum number of time steps. In the example, the 
conditionally stable Euler method required 405 time steps, as compared to about 
139 for the trapezoidal method, about three times as many. However, the Euler 
method is extremely easy to program and requires very little arithmetic per step. For 
preliminary investigations it is often the best method to use for mildly-stiff diffusion 
dominated problems. For refined investigations of such problems an explicit method 
of second order or higher, such as Adams-Bashforth or Runge-Kutta methods, is 
recommended. These explicit methods can be considered as effective mildly stiff- 
stable methods. However, it should be clear that as the degree of stiffness of the 
problem increases, the advantage begins to tilt towards implicit methods, as the 
reduced number of time steps begins to outweigh the increased cost per time step. 
The reader can repeat the above example with А; = —10, 000, А = —1, which is in 
the strongly-stiff category. 

'There is yet another technique for coping with certain stiff systems in fluid dynamic 
applications. 'This is known as the multigrid method. It has enjoyed remarkable 
success in many practical problems; however, we need an introduction to the theory 
of relaxation before it can be presented. 


8.6 Steady Problems 


In Chapter 6 we wrote the OAE solution in terms of the principal and spurious roots 
as follows: 














Un = “(ОТ m1 4s (08) Xm Hi Ceu (03)? tut PS. 
+¢12(01)3 9i +--+ +m2(Om)> Tm +++ Cus (ом)3 м 
-Feis(01)2 1 + +--+ na (05) fm +++: + смз(см)" 2м 
+etc., if there are more spurious roots (8.8) 


When solving a steady problem, we have no interest whatsoever in the transient por- 
tion of the solution. Our sole goal is to eliminate it as quickly as possible. Therefore, 
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the choice of a time-marching method for a steady problem is similar to that for a 
stiff problem, the difference being that the order of accuracy is irrelevant. Hence the 
explicit Euler method is a candidate for steady diffusion dominated problems, and 
the fourth-order Runge-Kutta method is a candidate for steady convection dominated 
problems, because of their stability properties. Among implicit methods, the implicit 
Euler method is the obvious choice for steady problems. 

When we seek only the steady solution, all of the eigenvalues can be considered to 
be parasitic. Referring to Fig. 8.1, none of the eigenvalues are required to fall in the 
accurate region of the time-marching method. Therefore the time step can be chosen 
to eliminate the transient as quickly as possible with no regard for time accuracy. 
For example, when using the implicit Euler method with local time linearization, Eq. 
6.96, one would like to take the limit h — oo, which leads to Newton's method, Eq. 
6.98. However, a finite time step may be required until the solution is somewhat close 
to the steady solution. 


8.7 Problems 


1. Repeat the time-march comparisons for diffusion (Section 8.4.2) and periodic 
convection (Section 8.4.3) using 2nd- and 3rd-order Runge-Kutta methods. 


2. Repeat the time-march comparisons for diffusion (Section 8.4.2) and periodic 
convection (Section 8.4.3) using the 3rd- and 4th-order Adams-Bashforth meth- 
ods. Considering the stability bounds for these methods (see problem 4 in 
Chapter 7) as well as their memory requirements, compare and contrast them 
with the 3rd- and 4th-order Runge-Kutta methods. 


3. Consider the diffusion equation (with v — 1) discretized using 2nd-order central 
differences on a grid with 10 (interior) points. Find and plot the eigenvalues 
and the corresponding modified wavenumbers. If we use the explicit Euler time- 
marching method what is the maximum allowable time step if all but the first 
two eigenvectors are considered parasitic? Assume that sufficient accuracy is 
obtained as long as |Ah| < 0.1. What is the maximum allowable time step if all 
but the first eigenvector are considered parasitic? 
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Chapter 9 


RELAXATION METHODS 


In the past three chapters, we developed a methodology for designing, analyzing, 
and choosing time-marching methods. These methods can be used to compute the 
time-accurate solution to linear and nonlinear systems of ODE's in the general form 


ай a 
— = F(ut 9.1 
= FL) (9.1) 


which arise after spatial discretization of a PDE. Alternatively, they can be used to 
solve for the steady solution of Eq. 9.1, which satisfies the following coupled system 


of nonlinear algebraic equations: 


F(a) =0 (9.2) 
In the latter case, the unsteady equations are integrated until the solution converges 


to a steady solution. The same approach permits a time-marching method to be used 
to solve a linear system of algebraic equations in the form 


At = Б (9.3) 


To solve this system using a time-marching method, a time derivative is introduced 
as follows 

ах? = 

— —Ax-b 9.4 

di (9.4) 
and the system is integrated in time until the transient has decayed to a sufficiently 
low level. Following a time-dependent path to steady state is possible only if all of 
the eigenvalues of the matrix А (or — А) have real parts lying in the left half-plane. 
Although the solution 2 = A^ !b exists as long as A is nonsingular, the ODE given by 
Eq. 9.4 has a stable steady solution only if A meets the above condition. 
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'The common feature of all time-marching methods is that they are at least first- 
order accurate. In this chapter, we consider iterative methods which are not time 
accurate at all. Such methods are known as relazation methods. While they are 
applicable to coupled systems of nonlinear algebraic equations in the form of Eq. 9.2, 
our analysis will focus on their application to large sparse linear systems of equations 
in the form 


Аки — }„=0 (9.5) 


where A, is nonsingular, and the use of the subscript b will become clear shortly. Such 
systems of equations arise, for example, at each time step of an implicit time-marching 
method or at each iteration of Newton's method. Using an iterative method, we seek 
to obtain rapidly a solution which is arbitrarily close to the exact solution of Eq. 9.5, 
which is given by 


Use Б Aj f, (9.6) 


9.1 Formulation of the Model Problem 


9.1.1 Preconditioning the Basic Matrix 


It is standard practice in applying relaxation procedures to precondition the basic 
equation. This preconditioning has the effect of multiplying Eq. 9.5 from the left 
by some nonsingular matrix. In the simplest possible case the conditioning matrix 
is a diagonal matrix composed of a constant D(b). If we designate the conditioning 
matrix by C, the problem becomes one of solving for u in 


CA — Cf, =0 (9.7) 


Notice that the solution of Eq. 9.7 is 


и = [CA] "Cf, = AC Cf, = Aj fs (9.8) 


which is identical to the solution of Eq. 9.5, provided C^! exists. 

In the following we will see that our approach to the iterative solution of Eq. 9.7 
depends crucially on the eigenvalue and eigenvector structure of the matrix C Aj, and, 
equally important, does not depend at all on the eigensystem of the basic matrix Ap. 
For example, there are well-known techniques for accelerating relaxation schemes if 
the eigenvalues of C A; are all real and of the same sign. To use these schemes, the 
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conditioning matrix C must be chosen such that this requirement is satisfied. A 
choice of C which ensures this condition is the negative transpose of Ap. 

For example, consider a spatial discretization of the linear convection equation 
using centered differences with a Dirichlet condition on the left side and no constraint 
on the right side. Using a first-order backward difference on the right side (as in 
Section 3.6), this leads to the approximation 


0 1 —Ua 
j = ed 0 
ó,u = —— Ep. 0 1 и+| 0 (9.9) 
_. ewe. ЧА 0 
=) 0 


The matrix in Eq. 9.9 has eigenvalues whose imaginary parts are much larger than 
their real parts. It can first be conditioned so that the modulus of each element is 1. 
This is accomplished using a diagonal preconditioning matrix 


10000 
01000 

D=2Ar| 00100 (9.10) 
00010 
00004 


which scales each row. We then further condition with multiplication by the negative 
transpose. The result is 


A» = —ATA, = 
0 1 1 
-1 0 1 —1 1 
=} 1 —1 0 1 
—1 —l. 0 1 
2 —1 1 
zd 0 1 
—2 1 
=| 1 0—2 0 1 (9.11) 
1 0 — 1 
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If we define a permutation matrix P t and carry out the process PT[— АТА, |Р (which 
just reorders the elements of А; and doesn't change the eigenvalues) we find 


01000][-1 0 1 00001 
00010 Oo. cur al 10000 
00001 1 0-2 0 1[|[]00010 
00100 1 0-2 1]//01000 
10000 D 1—2||00100 

epu 

г = v 

= bom 3 (9.12) 
jm. 1 


which has all negative real eigenvalues, as given in Appendix B. Thus even when 
the basic matrix A, has nearly imaginary eigenvalues, the conditioned matrix — A7 A, 
is nevertheless symmetric negative definite (i.e., symmetric with negative real eigen- 
values), and the classical relaxation methods can be applied. We do not necessarily 
recommend the use of — A? as a preconditioner; we simply wish to show that a broad 
range of matrices can be preconditioned into a form suitable for our analysis. 


9.1.2 The Model Equations 


Preconditioning processes such as those described in the last section allow us to 
prepare our algebraic equations in advance so that certain eigenstructures are guar- 
anteed. In the remainder of this chapter, we will thoroughly investigate some simple 
equations which model these structures. We will consider the preconditioned system 
of equations having the form 


> 


Aó — f 20 (9.13) 


where A is symmetric negative definite.? The symbol for the dependent variable has 
been changed to ф as a reminder that the physics being modeled is no longer time 


1А permutation matrix (defined as a matrix with exactly one 1 in each row and column and has 
the property that РТ = P-!) just rearranges the rows and columns of a matrix. 

?We use a symmetric negative definite matrix to simplify certain aspects of our analysis. Relax- 
ation methods are applicable to more general matrices. The classical methods will usually converge 
if A, is diagonally dominant, as defined in Appendix A. 
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accurate when we later deal with ODE formulations. Note that the solution of Eq. 
9.13, ф = A`! f, is guaranteed to exist because A is nonsingular. In the notation of 
Eqs. 9.5 and 9.7, 


A=CA, and f=Cf, (9.14) 


The above was written to treat the general case. It is instructive in formulating the 
concepts to consider the special case given by the diffusion equation in one dimension 
with unit diffusion coefficient v: 


леа g(x) (9.15) 


This has the steady-state solution 


2и = g(a) (9.16) 


which is the one-dimensional form of the Poisson equation. Introducing the three- 
point central differencing scheme for the second derivative with Dirichlet boundary 
conditions, we find 


— = —,B(1,-2,1 — 1 


where (bc) contains the boundary conditions and g contains the values of the source 
term at the grid nodes. In this case 


1 
А = xuB(.-2) 


fo 
Choosing C = Az?I, we obtain 


(9.18) 


Il 
© 
| 
м 
e 
al 
М.У 


where f = Az? fy. If we consider a Dirichlet boundary condition on the left side and 
either a Dirichlet or a Neumann condition on the right side, then A has the form 


A= B(1,6,1) 
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JU m Ed 
ehe ог —1 (9.20) 
Note that s = —1 is easily obtained from the matrix resulting from the Neumann 


boundary condition given in Eq. 3.24 using a diagonal conditioning matrix. А tremen- 
dous amount of insight to the basic features of relaxation is gained by an appropriate 
study of the one-dimensional case, and much of the remaining material is devoted to 
this case. We attempt to do this in such a way, however, that it is directly applicable 
to two- and three-dimensional problems. 


9.2 Classical Relaxation 


9.2.1 The Delta Form of an Iterative Scheme 


We will consider relaxation methods which can be expressed in the following delta 
form: 


where H is some nonsingular matrix which depends upon the iterative method. The 
matrix H is independent of n for stationary methods and is a function of n for 


nonstationary ones. ‘The iteration count is designated by the subscript n or the 
superscript (n). The converged solution is designated @,, so that 


бы = А7 (9.22) 


9.2.2 The Converged Solution, the Residual, and the Error 


> 


Solving Eq. 9.21 for @,,,, gives 
Ф, = П Н-144, – Hf = Gd, — HOF (9.23) 
where 
G=I+H"'A (9.24) 


Hence it is clear that H should lead to a system of equations which is easy to solve, 
or at least easier to solve than the original system. The error at the nth iteration is 
defined as 


> 


еп 


On ix Doo = On = А71 (9.25) 
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where Doo was defined in Eq. 9.22. The residual at the nth iteration is defined as 
Ta = Ad, —f (9.26) 


Multiply Eq. 9.25 by A from the left, and use the definition in Eq. 9.26. There results 
the relation between the error and the residual 


Ae, — Т = 0 (9.27) 


Finally, it is not difficult to show that 


eng = Gen (9.28) 
Consequently, G is referred to as the basic iteration matrix, and its eigenvalues, which 
we designate as am, determine the convergence rate of a method. 

In all of the above, we have considered only what are usually referred to as sta- 
tionary processes in which H is constant throughout the iterations. Nonstationary 
processes in which H (and possibly C) is varied at each iteration are discussed in 
Section 9.5. 


9.2.3 The Classical Methods 
Point Operator Schemes in One Dimension 


Let us consider three classical relaxation procedures for our model equation 


as given in Section 9.1.2. The Point-Jacobi method is expressed in point operator 
form for the one-dimensional case as 


TL 1 T, TL 
9f? = z e h- f (9.30) 
'This operator comes about by choosing the value of gone such that together with 
the old values of ф;1 and $;41, the jth row of Eq. 9.29 is satisfied. The Gauss-Seidel 
method is 





nd 1 n n 
of? = [фут of - fil (9:80) 


This operator is a simple extension of the point-Jacobi method which uses the most 
recent update of ф;_1. Hence the jth row of Eq. 9.29 is satisfied using the new values of 
Фф; and ф;—1 and the old value of фут. The method of successive overrelaxation (SOR) 
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is based on the idea that if the correction produced by the Gauss-Seidel method tends 
to move the solution toward @,,, then perhaps it would be better to move further in 
this direction. It is usually expressed in two steps as 


i 1 n+1 n 
à = ge ee t 
(P = oP uló; – ef" (9.32) 
where ш generally lies between 1 and 2, but it can also be written in the single line 


W 


n n n W n 
9f? = EEH- oP + 255 - 2:5 (89) 


4 2 
The General Form 


The general form of the classical methods is obtained by splitting the matrix A in 
Eq. 9.13 into its diagonal, D, the portion of the matrix below the diagonal, L, and 
the portion above the diagonal, U, such that 


A=L+D+U (9.34) 


Then the point-Jacobi method is obtained with H = —D, which certainly meets 
the criterion that it is easy to solve. The Gauss-Seidel method is obtained with 
H = —(L + D), which is also easy to solve, being lower triangular. 


9.3 The ODE Approach to Classical Relaxation 


9.3.1 The Ordinary Differential Equation Formulation 


The particular type of delta form given by Eq. 9.21 leads to an interpretation of 
relaxation methods in terms of solution techniques for coupled first-order ODE's, 
about which we have already learned a great deal. One can easily see that Eq. 9.21 
results from the application of the explicit Euler time-marching method (with h = 1) 
to the following system of ODE's: 


H— = Аф— f (9.35) 
This is equivalent to 


T = HO Ab А) = n7 lb - f (9.36) 
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In the special case where H^! A depends on neither u nor t, Hof is also independent 
of t, and the eigenvectors of H~'A are linearly independent, the solution can be 
written as 
ф = пећ +- + eye au t. (9.37) 
—— M Á—— 
error 


where what is referred to in time-accurate analysis as the transient solution, is now 
referred to in relaxation analysis as the error. It is clear that, if all of the eigenvalues 
of H^! A have negative real parts (which implies that H^! A is nonsingular), then the 
system of ОРЕ? has a steady-state solution which is approached as t — oo, given by 


Bo. = А} (9.38) 


which is the solution of Eq. 9.13. We see that the goal of a relaxation method is to 
remove the transient solution from the general solution in the most efficient way pos- 
sible. The A eigenvalues are fixed by the basic matrix in Eq. 9.36, the preconditioning 
matrix in 9.7, and the secondary conditioning matrix in 9.35. The c eigenvalues are 
fixed for a given Ah by the choice of time-marching method. Throughout the remain- 
ing discussion we will refer to the independent variable t as “time”, even though no 
true time accuracy is involved. 

In a stationary method, H and C in Eq. 9.36 are independent of t, that is, they 
are not changed throughout the iteration process. The generalization of this in our 
approach is to make h, the “time” step, a constant for the entire iteration. 

Suppose the explicit Euler method is used for the time integration. For this method 
Om = 1 + àmh. Hence the numerical solution after n steps of a stationary relaxation 
method can be expressed as (see Eq. 6.28) 


puce pub b dar cba) Spes E Ow АЧ 209.89) 
————————————————————————————————————————————— 
error 


The initial amplitudes of the eigenvectors are given by the magnitudes of the c. 
These are fixed by the initial guess. In general it is assumed that any or all of the 
eigenvectors could have been given an equally *bad" excitation by the initial guess, 
so that we must devise a way to remove them all from the general solution on an 
equal basis. Assuming that H^! A has been chosen (that is, an iteration process has 
been decided upon), the only free choice remaining to accelerate the removal of the 
error terms is the choice of h. As we shall see, the three classical methods have all 
been conditioned by the choice of Н to have an optimum A equal to 1 for a stationary 
iteration process. 
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9.3.2 ODE Form of the Classical Methods 


The three iterative procedures defined by Eqs. 9.30, 9.31 and 9.32 obey no apparent 
pattern except that they are easy to implement in a computer code since all of the 
data required to update the value of one point are explicitly available at the time 
of the update. Now let us study these methods as subsets of ODE as formulated in 
Section 9.3.1. Insert the model equation 9.29 into the ODE form 9.35. Then 


ut = B(1,-2,)4- f (9.40) 
As a start, let us use for the numerical integration the explicit Euler method 
Pn+1 = on + Аф, (9.41) 
with a step size, h, equal to 1. We arrive at 
H($, =) = B0, 2,98, - f (9.42) 


It is clear that the best choice of H from the point of view of matrix algebra is 
—B(1, —2,1) since then multiplication from the left by — B^! (1, —2,1) gives the cor- 
rect answer in one step. However, this is not in the spirit of our study, since multi- 
plication by the inverse amounts to solving the problem by a direct method without 
iteration. The constraint on Н that is in keeping with the formulation of the three 
methods described in Section 9.2.3 is that all the elements above the diagonal (or 
below the diagonal if the sweeps are from right to left) are zero. If we impose this 
constraint and further restrict ourselves to banded tridiagonals with a single constant 
in each band, we are led to 


> > 


B(-8, =, 0 ($an — &) = BOL 2,08, - 7 (9.43) 


where 8 and w are arbitrary. With this choice of notation the three methods presented 
in Section 9.2.3 can be identified using the entries in Table 9.1. 


TABLE 9.1: VALUES OF 8. and w IN EQ. 9.43 THAT 
LEAD TO CLASSICAL RELAXATION METHODS 


Method Equation 


Point-Jacobi 6.2.3 
1 Gauss-Seidel 6.2.4 
2/1 + sin (a7 T т) Optimum SOR 6.2.5 
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'The fact that the values in the tables lead to the methods indicated can be verified 
by simple algebraic manipulation. However, our purpose is to examine the whole 
procedure as a special subset of the theory of ordinary differential equations. In this 
light, the three methods are all contained in the following set of ODE’s 


dd 2 TEES 
T = В (—8,—,0) [в(1,—2,1)8— f| (9.44) 


and appear from it in the special case when the explicit Euler method is used for its 
numerical integration. The point operator that results from the use of the explicit 
Euler scheme is 


ic (Eor S EIU E aii) - ((wh = 1)9”) + (Ea) — T f;(9.45) 


This represents a generalization of the classical relaxation techniques. 


9.4 Eigensystems of the Classical Methods 


The ODE approach to relaxation can be summarized as follows. The basic equation 
to be solved came from some time-accurate derivation 


Ayu — f, =0 (9.46) 
This equation is preconditioned in some manner which has the effect of multiplication 


by a conditioning matrix C giving 


> 


Ag — f =0 (9.47) 


An iterative scheme is developed to find the converged, or steady-state, solution of 
the set of ODE's 


H— = A$- f (9.48) 


This solution has the analytical form 


> 


bn =€n + Goo (9.49) 


where En is the transient, or error, and Doo = Af is the steady-state solution. The 
three classical methods, Point-Jacobi, Gauss-Seidel, and SOR, are identified for the 
one-dimensional case by Eq. 9.44 and Table 9.1. 
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Given our assumption that the component of the error associated with each eigen- 
vector is equally likely to be excited, the asymptotic convergence rate is determined 
by the eigenvalue om of G (= I + H^! A) having maximum absolute value. Thus 


Convergence rate ^ |с , M=1,2,---,M (9.50) 


In this section, we use the ODE analysis to find the convergence rates of the three 
classical methods represented by Eqs. 9.30, 9.31, and 9.32. It is also instructive to 
inspect the eigenvectors and eigenvalues in the H^! A matrix for the three methods. 
This amounts to solving the generalized eigenvalue problem 


Atm = ÀmH im (9.51) 


for the special case 
B(1, 2, 1)z,, = AmB(—B, £, 0)£m (9.52) 
w 


The generalized eigensystem for simple tridigonals is given in Appendix B.2. The 
three special cases considered below are obtained with a = 1, b = —2, c = 1, d = — 8, 
e = 2/w, and f = 0. To illustrate the behavior, we take M = 5 for the matrix order. 
This special case makes the general result quite clear. 


9.4.1 The Point-Jacobi System 
If 8 = 0 and w = 1 in Eq. 9.44, the ODE matrix H^! A reduces to simply BOS, —1, D. 


The eigensystem can be determined from Appendix B.1 since both d and f are zero. 
The eigenvalues are given by the equation 





Am = —1 + cos ( ac 


=1,2,...,M 9.53 
x) m TSA ? ( ) 


The A-o relation for the explicit Euler method is om = 1 + Amh. This relation can 
be plotted for any h. The plot for h — 1, the optimum stationary case, is shown in 
Fig. 9.1. For h < 1, the maximum |o,,| is obtained with m = 1, Fig. 9.2 and for 
h > 1, the maximum |o,,| is obtained with m = M, Fig. 9.3. Note that for h > 1.0 
(depending оп M) there is the possibility of instability, i.e. |o;,| > 1.0. To obtain the 
optimal scheme we wish to minimize the maximum |o;,| which occurs when h = 1, 
|o1| = |o. | and the best possible convergence rate is achieved: 


lo, lu, = сов ( Ez -) (9.54) 
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For M = 40, we obtain |o;,|,,,,, = 0.9971. Thus after 500 iterations the error content 
associated with each eigenvector is reduced to no more than 0.23 times its initial level. 
Again from Appendix B.1, the eigenvectors of НА are given by 


э у mr 
£j = (7) = sin 1( 


——— |j —1,2,..., M 9.55 
um) FHLB (9.55) 


This is a very “well-behaved” eigensystem with linearly independent eigenvectors and 
distinct eigenvalues. The first 5 eigenvectors are simple sine waves. For M = 5, the 
eigenvectors can be written as 


1/2 3/2 1 
3/2 V3/2 0 
dj = 1 ‚92 = 0 ЕА —1 ^ 
3/2 — 3/2 0 
1/2 — 3/2 1 
3/2 1/2 
—V3/2 -V3/2 
T4 = 0 ‚ ®5 = 1 (9.56) 
v3/2 –у3/2 
—/3/2 1/2 
The corresponding eigenvalues are, from Eq. 9.53 
ee STS v3 = —0.134--- 
`= -1+5 --05 
Аз = —1 = —]1.0 (9.57) 
м= -l-4 =-15 


№ = ЕЕЕ = —1.866--- 


From Eq. 9.39, the numerical solution written in full is 


n-o = є[ї—(1—-—)Һ"2, 
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Values are equal 


lon! 





.0 


Figure 9.1: The о, А relation for Point-Jacobi, h = 1, M = 5. 


+ єз[1 a (1 JA|"za 





PEE DE? 
+ о 6П1—(1+ Vas, (9.58) 


9.4.2 The Gauss-Seidel System 


If 8 and ш are equal to 1 in Eq. 9.44, the matrix eigensystem evolves from the relation 
B(1, —2, 1)2m = AmB(—1, 2, 0)2m (9.59) 


which can be studied using the results in Appendix B.2. One can show that the H~!A 
matrix for the Gauss-Seidel method, Ags , is 


Ags = B(-1,2,0)B(1,-2,1) = 


9.4. EIGENSYSTEMS OF THE CLASSICAL METHODS 177 


1.0 Approaches 1.0 





-2.0 -1.0 0.0 


gh 


Exceeds 1.0 at h = 1.072 / Ustable h > 1.072 


| o. 





Figure 9.3: The о, А relation for Point-Jacobi, h = 1.1, M = 5. 
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—1 1/2 1 
@ -=3/4, “472 2 
0 1/8 —3/4 1/2 3 
0 1/16 1/8 —3/4 1/2 4 (9.60) 
0 1/32 1/16 1/8 -—3/4 1/2 5 
Qc DM" aan M 


The eigenvector structure of the Gauss-Seidel ODE matrix is quite interesting. If M 
is odd there are (M 4- 1)/2 distinct eigenvalues with corresponding linearly indepen- 
dent eigenvectors, and there are (M — 1)/2 defective eigenvalues with corresponding 
principal vectors. The equation for the nondefective eigenvalues in the ODE matrix 
is (for odd M) 

















тт М +1 
ышк A —1,2,... 9.61 
ов (Im). m-12324275 (9.61) 
and the corresponding eigenvectors are given by 
3 ( тт J^. А J тит ) 19 M+1 (9.62) 
Lm |cos si = xs : 
m М +1 n J М +1 ? m ? ? ? 9 








The A-o relation for h = 1, the optimum stationary case, is shown in Fig. 9.4. The 
Om With the largest amplitude is obtained with m = 1. Hence the convergence rate is 


cos (x -)] | (9.63) 


Since this is the square of that obtained for the Point-Jacobi method, the error associ- 
ated with the ^worst" eigenvector is removed in half as many iterations. For M — 40, 
lOmlmar = 0.9942. 250 iterations are required to reduce the error component of the 
worst eigenvector by a factor of roughly 0.23. 

The eigenvectors are quite unlike the Point-Jacobi set. They are no longer sym- 
metrical, producing waves that are higher in amplitude on one side (the updated side) 
than they are on the other. Furthermore, they do not represent a common family for 
different values of M. 

The Jordan canonical form for M = 5 is 





l9 m [az Ed 





X= Ane X = Jas = | bs 1 | (9.64) 
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-2.0 -1.0 0.0 
A h 


Figure 9.4: The c, А relation for Gauss-Seidel, h = 1.0, M = 5. 


The eigenvectors and principal vectors are all real. For M = 5 they can be written 


[oq үз, и e о) 


_ 3/4 | | V3/4 Ol) = Deol 0 
21 = 3/4 ‚Хә = 0 ‚Хз = 0 ‚4 = —1 ‚5 = 4 (9.65) 
9/16 — 3/16 0 0 —4 
9/32 — 3/32 0 0 1 
The corresponding eigenvalues are 
At = —1/4 
№ = —3/4 
Аз = —1 
(4) | Defective, linked to (9.66) 
(5) Аз Jordan block 
'The numerical solution written in full is thus 
> > h > 
Pn Ф = all- т)" 
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=] E 
+ ш h)” + eh = aa + eh = = hye? T3 


$ laa = h)” + oh (1 В)", 


seb dv (9.67) 


9.4.3 The SOR System 


If 8 = 1 and 2/w = х in Eq. 9.44, the ODE matrix is B^!(—1,2,0) B(1, —2, 1). 
One can show that this can be written in the form given below for M — 5. The 
generalization to any M is fairly clear. The H-!A matrix for the SOR method, 
Asor = B-\(-1,2, 0)B(1, —2, 1), is 





—2x4 x^ 0 0 0 
—249 + a4 z? — 2r" x 0 0 
— | 22? +x? 2? – 20% +12 — x? — 2x 1 0 (9.68) 
$ —2xr +r? xr? +r? quiu 2r? +r’ xz? — 2x4 х“ 
—2 +7 ]e9»d m^ qp +r? r? OP +r’ х%— 24° 
Eigenvalues of the system are given by 
2 
Am = 1 (SUI ее (9.69) 
where 
zm = [4(1—-ш)+ wp? 1? 


Pm cos|mr/(M + 1)] 


If w = 1, the system is Gauss-Seidel. If 4(1 — w) +w*pm < 0, Zm and Àm are complex. 
If w is chosen such that 4(1 — w) + wp? = 0,w is optimum for the stationary case, 
and the following conditions hold: 


1. Two eigenvalues are real, equal and defective. 


2. If M is even, the remaining eigenvalues are complex and occur in conjugate 
pairs. 


3. If M is odd, one of the remaining eigenvalues is real and the others are complex 
occurring in conjugate pairs. 
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One can easily show that the optimum ш for the stationary case is 














, T 
opt = 2/ |1 + sin (xxi) (9.70) 
and for w = Wopt 
Neg гел 
Im = Ci! sin ie -)] (9.71) 
where 
Wo M rs 
Cm = 2 Pm t pi = 7, 





Using the explicit Euler method to integrate ће ODE’s, om = 1—h+h¢?, and if 
h = 1, the optimum value for the stationary case, the А-с relation reduces to that 
shown in Fig. 9.5. This illustrates the fact that for optimum stationary SOR all the 
|o| are identical and equal to vj; — 1. Hence the convergence rate is 


One = CAI 1 + sin ( 





ire) 


For M = 40, |om| mas = 0.8578. Hence the worst error component is reduced to less 
than 0.23 times its initial value in only 10 iterations, much faster than both Gauss- 
Seidel and Point-Jacobi. In practical applications, the optimum value of ш may have 
to be determined by trial and error, and the benefit may not be as great. 

For odd M, there are two real eigenvectors and one real principal vector. The 
remaining linearly independent eigenvectors are all complex. For M — 5 they can be 
written 


1/2 —6 V3 )/2 1 
1/2 9 V3(1+ iv2)/6 | - 0 
ту= | 1/3 |,z2= | 16 |,£34= 0 ‚®5 = | 1/3 | (9.73) 
1/6 13 МЗ(5 + i/2)/54 0 
1/18 6 V3(7 + 4iy/2)/162 1/9 


The corresponding eigenvalues are 


A. жек DS 

(2) Defective linked to А! 
№ = —(10—2V2i)/9 

м = —(10+2/2%)/9 


№ = 4/3 (9.74) 
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lo. 





Aah 


Figure 9.5: The о, А relation for optimum stationary SOR, M = 5, h = 1. 


The numerical solution written in full is 


On — boo = [a(1—2h/3)" + conh(1 — 2h/3)^-]z, 

+ (1 — 2h/3)"z; 

+ exl = (10 — 2V2i)h/9]"2x3 

+ afl- (10+ 2V2i)h/9]"24 

+ es(1— 4h/3)" zs (9.75) 





9.5 Nonstationary Processes 


In classical terminology a method is said to be nonstationary if the conditioning 
matrices, Н and C, are varied at each time step. This does not change the steady- 
state solution Aj! fp but it can greatly affect the convergence rate. In our ODE 
approach this could also be considered and would lead to a study of equations with 
nonconstant coefficients. It is much simpler, however, to study the case of fixed 
Н and C but variable step size, Л. This process changes the Point-Jacobi method 
to Richardson's method in standard terminology. For the Gauss-Seidel and SOR 
methods it leads to processes that can be superior to the stationary methods. 
The nonstationary form of Eq. 9.39 is 
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N N 
Фу = e$ (1+ Ahn) +--+ сл ЦО + Amhn) 
n=1 n=1 
> N > 
+ + смтм ЦО + Ам) + Фф» (9.76) 


n—l 


where the symbol II stands for product. Since h, can now be changed at each step, 
the error term can theoretically be completely eliminated in M steps by taking Am = 


—1/Am, for m = 1,2,---, M. However, the eigenvalues Àm are generally unknown and 
costly to compute. It is therefore unnecessary and impractical to set hm = —1/Am 
for m = 1,2,..., M. We will see that a few well chosen h’s can reduce whole clusters 


of eigenvectors associated with nearby A's in the Àm spectrum. This leads to the 
concept of selectively annihilating clusters of eigenvectors from the error terms as 
part of a total iteration process. This is the basis for the multigrid methods discussed 
in Chapter 10. 

Let us consider the very important case when all of the Àm are real and nega- 
tive (remember that they arise from a conditioned matrix so this constraint is not 
unrealistic for quite practical cases). Consider one of the error terms taken from 


M N 
en = фу = Qu = 5 Gaius П (1 + Amhn) (9.77) 
т=1 n=1 
and write it in the form 
_ М 
Omm Fem) = Cmm П (1+ Ah.) (9.78) 
mel 


where P, signifies an “Euler” polynomial. Now focus attention on the polynomial 
(Pe) y (4) = (1+ hA) + АА) (1 + А) (9.79) 


treating it as a continuous function of the independent variable А. In the annihilation 
process mentioned after Eq. 9.76, we considered making the error exactly zero by 
taking advantage of some knowledge about the discrete values of Am for a particular 
case. Now we pose a less demanding problem. Let us choose the h, so that the 
maximum value of (Рь) (А) is as small as possible for all А lying between A, and A, 
such that Ay < A € A, € 0. Mathematically stated, we seek 


ME \(P-)w(A)| = minimum , with(P.)v(0) = 1 (9.80) 
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'This problem has a well known solution due to Markov. It is 


(2% А м) 
(Р.)л(А) = о aa ete (9.81) 
where 
Tw(y) = cos(N arccos y) (9.82) 


are the Chebyshev polynomials along the interval —1 < y < 1 and 


TEE y- yP] (9.83) 


are the Chebyshev polynomials for |y| > 1. In relaxation terminology this is generally 
referred to as Richardson’s method, and it leads to the nonstationary step size choice 
given by 


1 


Тубу) = 5 











+= eN м)» EEN, n=1,2,...N (9.84) 


Remember that all А are negative real numbers representing the magnitudes of Am in 
an eigenvalue spectrum. 

The error in the relaxation process represented by Eq. 9.76 is expressed in terms 
of a set of eigenvectors, Tm, amplified by the coefficients cm П(1 + Amhn). With each 
eigenvector there is a corresponding eigenvalue. Eq. 9.84 gives us the best choice of a 
series of h, that will minimize the amplitude of the error carried in the eigenvectors 
associated with the eigenvalues between A, and A,. 

As an example for the use of Eq. 9.84, let us consider the following problem: 


Minimize the maximum error asso- 
ciated with the A eigenvalues in the (9.85) 


interval —2 < А € —1 using only З 
iterations. 





The three values of h which satisfy this problem are 


hn = 2/ Е — cos [e (9.86) 
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and the amplitude of the eigenvector is reduced to 
(Pe)3(A) = T3(2A + 3)/T3(3) (9.87) 
where 
T;(3) = {[3 + V8P + [3 — v8] }/2 = 99 (9.88) 


A plot of Eq. 9.87 is given in Fig. 9.6 and we see that the amplitudes of all the 
eigenvectors associated with the eigenvalues in the range —2 € А € —1 have been 
reduced to less than about 1% of their initial values. The values of h used in Fig. 9.6 
are 


hy = 4/(6— V3) 
hg = А/(6—0) 
hs = 4/(6+ v3) 
Return now to Eq. 9.76. This was derived from Eq. 9.37 on the condition that the 


explicit Euler method, Eq. 9.41, was used to integrate the basic ODE’s. If instead 
the implicit trapezoidal rule 


1 
Ọn+1 = Pn + ДС + Gn) (9.89) 


is used, the nonstationary formula 


1 
_ м , N [1- g nÀm » 
On = SE Cmm П SS == р Poo (9.90) 
m=1 n=1\ 1 — 5! nÀm 


would result. This calls for a study of the rational “trapezoidal” polynomial, P: 


1 
(P)N() = [I | —d— (9.91) 
n=l \ ] — 5А 


under the same constraints as before, namely that 


me (СР) м(А)| = minimum, (9.92) 


with (Р,)х(0) = 1 
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Figure 9.6: Richardson’s method for 3 steps, minimization over —2 < A < —1. 
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The optimum values of h can also be found for this problem, but we settle here for 
the approximation suggested by Wachspress 


9 A (n-1)/(N -1) 
nA | os cub o een 9.93 
[Ж (ss) , n 94) Í ( ) 


This process is also applied to problem 9.85. The results for (P,)3(A) are shown in 
Fig. 9.7. The error amplitude is about 1/5 of that found for (P.)3(A) in the same 
interval of A. The values of h used in Fig. 9.7 are 


h = 1 
һә = у 
ha = 90 


9.6 Problems 
1. Given a relaxation method in the form 
НАф, = Аф, — f 
show that 
bn = G” $o + (I - G")A" f 
where G = I + НА. 
2. For a linear system of the form (A; + Ao)z = b, consider the iterative method 


(Т uA1)8 = (I — pA2)£n + ub 
(I + pA2)£n41 = (1 — нА) + ub 





where u is a parameter. Show that this iterative method can be written in the 
form 


H(xyai = Lr) = (Ai + Ag) Xx —b 
Determine the iteration matrix С if u = —1/2. 


3. Using Appendix В.2, find the eigenvalues of НА for the SOR method with 
A = B(4:1,—2,1) and w = шош. (You do not have to find H^!. Recall that 
the eigenvalues of H^!A satisfy Az,, = АН.) Find the numerical values, 
not just the expressions. Then find the corresponding |с, | values. 


188 CHAPTER 9. RELAXATION METHODS 














0.9 
0.8 
07 
0.6 
ra 
3 
d 0.5 
e 
ae 
a o4 
0.3 - 
0.2 
0.1 
ПСЕ onse ЕЕЕ. з > тоз т 
i 
-2 -1.8 -1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 




















Figure 9.7: Wachspress method for З steps, minimization over —2 < A < —1. 
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4. Solve the following equation on the domain 0 < x < 1 with boundary conditions 
и(0) = 0, u(1) = 1: 


ru en, 
— — 6% = 
дз? 
For the initial condition, use u(r) = 0. Use second-order centered differences 
on a grid with 40 cells (M — 39). Iterate to steady state using 
the point-Jacobi method, 
the Gauss-Seidel method, 
the SOR method with the optimum value of ш, and 


the 3-step Richardson method derived in Section 9.5. 


(a 
(b 
(c 
(d 


Ми М мМ мәе 


Plot the solution after the residual is reduced by 2, 3, and 4 orders of mag- 
nitude. Plot the logarithm of the L2-norm of the residual vs. the number of 
iterations. Determine the asymptotic convergence rate. Compare with the the- 
oretical asymptotic convergence rate. 
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Chapter 10 


MULTIGRID 


The idea of systematically using sets of coarser grids to accelerate the convergence of 
iterative schemes that arise from the numerical solution to partial differential equa- 
tions was made popular by the work of Brandt. There are many variations of the 
process and many viewpoints of the underlying theory. The viewpoint presented here 
is a natural extension of the concepts discussed in Chapter 9. 


10.1 Motivation 


10.1.1  Eigenvector and Eigenvalue Identification with Space 
Frequencies 


Consider the eigensystem of the model matrix B(1,—2,1). The eigenvalues and 
eigenvectors are given in Sections 4.3.2 and 4.3.3, respectively. Notice that as the 
magnitudes of the eigenvalues increase, the space-frequency (or wavenumber) of the 
corresponding eigenvectors also increase. That is, if the eigenvalues are ordered such 
that 


I S [Az] S --- < [Aml (10.1) 


then the corresponding eigenvectors are ordered from low to high space frequencies. 
This has a rational explanation from the origin of the banded matrix. Note that 


2 
22 sin(mz) = —m? sin(mz) (10.2) 
and recall that 
Ls. i AR: т 
1 pn _ zx ees - 10. 
быф = B0, -2, 1) = X |200), xj (10.3) 
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where D(X) is a diagonal matrix containing the eigenvalues. We have seen that 
Xto represents a sine transform, and Хд, a sine synthesis. Therefore, the opera- 


tion ALD) represents the numerical approximation of the multiplication of the 


appropriate sine wave by the negative square of its wavenumber, —m?. One finds 
that 








1 М +15? 
Am = ( = ) [—2 + 2cos ( ЕД 
TU 


— ду —m? M 10.4 
Ax? uu) m,m << (10.4) 


Hence, the correlation of large magnitudes of Am with high space-frequencies is to be 
expected for these particular matrix operators. This is consistent with the physics of 
diffusion as well. However, this correlation is not necessary in general. In fact, the 
complete counterexample of the above association is contained in the eigensystem 
for Bs, 1, D. For this matrix one finds, from Appendix B, exactly the opposite 
behavior. 


10.1.2 Properties of the Iterative Method 


The second key motivation for multigrid is the following: 


e Many iterative methods reduce error components corresponding to eigenvalues 
of large amplitude more effectively than those corresponding to eigenvalues of 
small amplitude. 


This is to be expected of an iterative method which is time accurate. It is also 
true, for example, of the Gauss-Seidel method and, by design, of the Richardson 
method described in Section 9.5. The classical point-Jacobi method does not share 
this property. As we saw in Section 9.4.1, this method produces the same value of |c | 
for Amin and Amar. However, the property can be restored by using h < 1, as shown 
in Fig. 9.2. 

When an iterative method with this property is applied to a matrix with the 
above correlation between the modulus of the eigenvalues and the space frequency of 
the eigenvectors, error components corresponding to high space frequencies will be 
reduced more quickly than those corresponding to low space frequencies. This is the 
key concept underlying the multigrid process. 


10.2 The Basic Process 


First of all we assume that the difference equations representing the basic partial 
differential equations are in a form that can be related to a matrix which has certain 


10.2. THE BASIC PROCESS 193 


basic properties. This form can be arrived at “naturally” by simply replacing the 
derivatives in the PDE with difference schemes, as in the example given by Eq. 3.27, 
or it can be “contrived” by further conditioning, as in the examples given by Eq. 9.11. 
The basic assumptions required for our description of the multigrid process are: 


1. The problem is linear. 
2. The eigenvalues, Am, of the matrix are all real and negative. 


3. The Am are fairly evenly distributed between their maximum and minimum 
values. 


4. ‘The eigenvectors associated with the eigenvalues having largest magnitudes can 
be correlated with high frequencies on the differencing mesh. 


5. The iterative procedure used greatly reduces the amplitudes of the eigenvectors 
associated with eigenvalues in the range between PIE and |A|mas- 


These conditions are sufficient to ensure the validity of the process described next. 

Having preconditioned (if necessary) the basic finite differencing scheme by a pro- 
cedure equivalent to the multiplication by a matrix C, we are led to the starting 
formulation 


С[4ф, — ЛЬ] = 0 (10.5) 


where the matrix formed by the product C A; has the properties given above. In Eq. 
10.5, the vector n represents the boundary conditions and the forcing function, if 
any, and Doo is a vector representing the desired exact solution. We start with some 
initial guess for Poo and proceed through n iterations making use of some iterative 
process that satisfies property 5 above. We do not attempt to develop an optimum 
procedure here, but for clarity we suppose that the three-step Richardson method 
illustrated in Fig. 9.6 is used. At the end of the three steps we find r, the residual, 
where 


r= C[Aió - fi] (10.6) 


Recall that the ó used to compute r is composed of the exact solution Doo and the 
error e in such a way that 


Ae—r=0 (10.7) 
where 


А= бА (10.8) 
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If one could solve Eq. 10.7 for e then 


> > 


фе=Ф—е (10.9) 


Thus our goal now is to solve for е. We can write the exact solution for e in terms of 
the eigenvectors of A, and the c eigenvalues of the Richardson process in the form: 


Ж 7 


Tx 


m [I let ha) + у: Cmim [[ [o(Amfn)] (10.10) 


m=M/2+1 n=1 


very low amplitude 


Combining our basic assumptions, we can be sure that the high frequency content of 
€ has been greatly reduced (about 196 or less of its original value in the initial guess). 
In addition, assumption 4 ensures that the error has been smoothed. 

Next we construct a permutation matrix which separates a vector into two parts, 
one containing the odd entries, and the other the even entries of the original vector 
(or any other appropriate sorting which is consistent with the interpolation approxi- 
mation to be discussed below). For a 7-point example 


e> | 0100000][e| 

ед 0001000 е» 

ев 0000010 ез Я 

ег |= 11000000 ед НЕ (10.11) 
ез 0010000 es ёо 

es 0000100 ев 

ет 000000 1 ez 


Multiply Eq. 10.7 from the left by P and, since a permutation matrix has an inverse 
which is its transpose, we can write 


PA[P-1P]e = Pr (10.12) 
The operation РАР! partitions the A matrix to form 
A, А + = 
| e | = | г | (10.13) 
Аз A4 
Notice that 


Ае, + Age, = Te (10.14) 
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is an exact expression. 

At this point we make our one crucial assumption. It is that there is some connec- 
tion between ee and е, brought about by the smoothing property of the Richardson 
relaxation procedure. Since the top half of the frequency spectrum has been removed, 
it is reasonable to suppose that the odd points are the average of the even points. 
For example 


Q 


1 
е1 5 (ea + ех) 


1 
ез œ~ (es e4) 
2 
1 — — 
€5 f£ 5 (ea + eg) or €o = Ае, (10.15) 
1 
ет 7X 3 (es + е) 


It is important to notice that ea and ер represent errors on the boundaries where the 
error is zero if the boundary conditions are given. It is also important to notice that we 
are dealing with the relation between e and r so the original boundary conditions and 
forcing function (which are contained in f in the basic formulation) no longer appear 
in the problem. Hence, no aliasing of these functions can occur in subsequent steps. 
Finally, notice that, in this formulation, the averaging of e is our only approximation, 
no operations on T are required or justified. 

If the boundary conditions are Dirichlet, e; and ер are zero, and one can write for 
the example case 


100 
ie 5 Ehe (10.16) 
00 1 
With this approximation Eq. 10.14 reduces to 
Ае, + ААе = Te (10.17) 
ог 
Ace, — Te = 0 (10.18) 
where 


A, = [Ai + Ao AL] (10.19) 
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The form of A,, the matrix on the coarse mesh, is completely determined by the 
choice of the permutation matrix and the interpolation approximation. If the original 
A had been В(7 : 1, —2, 1), our 7-point example would produce 


=) 1 1 
—2 1 1 
—2 а Л de 
PAP! = = (10.20) 
1 —2 diis x 
1 1 29 D 
ir ï —2 
1 —2 
and Eq. 10.18 gives 
5% 
А\ А» eer Ac 
—2 1 1 i : i —1 1/2 
—2 + L3 = | il =| V2 -1 1/2 | (10.21) 
—2 1 1 А 1/2 —1 


If the boundary conditions are mixed Dirichlet-Neumann, A in the 1-D model 
equation is B(1, 0, 1) where b= [-2, -2,..., -2, —1]". The eigensystem is given by 
Eq. B.19. It is easy to show that the high space-frequencies still correspond to the 
eigenvalues with high magnitudes, and, in fact, all of the properties given in Section 
10.1 are met. However, the eigenvector structure is different from that given in 
Eq. 9.55 for Dirichlet conditions. In the present case they are given by 





2m —1 
Lim = sin [ (a) > m= 1, 2, ia ‚М (10.22) 


and are illustrated in Fig. 10.1. All of them go through zero on the left (Dirichlet) 
side, and all of them reflect on the right (Neumann) side. 

For Neumann conditions, the interpolation formula in Eq. 10.15 must be changed. 
In the particular case illustrated in Fig. 10.1, e; is equal to em. If Neumann conditions 
are on the left, eg = ei. When e; = em, the example in Eq. 10.16 changes to 


А, = (10.23) 


1 
2 


Cc Eq 
CO =.=. © 
or о c 
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xL 


Figure 10.1: Eigenvectors for the mixed Dirichlet-Neumann case. 


The permutation matrix remains the same and both A; and A» in the partitioned 
matrix РАР! are unchanged (only A, is modified by putting —1 in the lower right 
element). Therefore, we can construct the coarse matrix from 


A3 
Ay A» —— Ac 
E, ld i | Ў zm 5 
29 + 11 iS ;1|[7|U2 -1. 1/2 | (1024) 
2) bat ; 1/2 —1/9 


which gives us what we might have “expected.” 

We will continue with Dirichlet boundary conditions for the remainder of this 
Section. At this stage, we have reduced the problem from B(1, —2, 1)е = г on the 
fine mesh to ls, —2, 1)e, = г, on the next coarser mesh. Recall that our goal is 
to solve for e, which will provide us with the solution Poo using Eq. 10.9. Given е, 
computed on the coarse grid (possibly using even coarser grids), we сап compute €, 
using Eq. 10.15, and thus е. In order to complete the process, we must now determine 
the relationship between бе апа е. 

In order to examine this relationship, we need to consider the eigensystems of A 
and A,: 


A=XAX™, Ae = ХХ} (10.25) 


For A = B(M : 1, —2, 1) the eigenvalues and eigenvectors are 


: тт уер, M 








ас 
n= — , m — Sin 
COS Mil x S 
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Based on our assumptions, the most difficult error mode to eliminate is that with 
m — 1, corresponding to 


0 [1 — cos (тт). zı = sin (т 


For example, with М = 51, А; = — 0.003649. If we restrict our attention to odd M, 
then M, = (M — 1) /2 is the size of Ae. The eigenvalue and eigenvector corresponding 
to m = 1 for the matrix A, = Í B(M., 1, —2, 1) are 


sae [om Gr 


For M = 51 (М, = 25), we obtain (А), = —0.007291 = 1.998\,. As М increases, 
(Ас), approaches 2A,. In addition, one can easily see that (ze); coincides with тү at 











)!. j=1,2,---,M_ (10.27) 


> 


, (ze) = sin 








2л 
j i= 1,2,- --, M, (10.28 
ill Aree Me ( ) 








every second point of the latter vector, that is, it contains the even elements of 11. 
Now let us consider the case in which all of the error consists of the eigenvector 
component 21, i.e., e = xı. Then the residual is 


> 


т = Ату = Ат (10.29) 
and the residual on the coarse grid is 
Te = Msn (10.30) 


since (z,), contains the even elements of тү. The exact solution on the coarse grid 
satisfies 


> 


e = A7 Te = ХАХА (Ze)1 


(10.31) 
1 
0 
— MXA! Я 
0 
(10.32) 
1/(Ac)1 
0 
= АХ, А 
0 


(10.33) 
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MN 
Ы 
(10.34) 
N LE) (10.35) 


Since our goal is to compute е = ті, in addition to interpolating e, to the fine grid 
(using Eq. 10.15), we must multiply the result by 2. This is equivalent to solving 


7575 
g Ac€e = (10.36) 
ог 
1 M 
zB(M. :1,2,1)& = T. (10.37) 


In our case, the matrix A — B(M : 1, —2, 1) comes from a discretization of the 
diffusion equation, which gives 


V 


А= дт 


B(M : 1,2,1) (10.38) 
and the preconditioning matrix C is simply 


^ 2 
СЕЕ а (10.39) 
и 
Applying the discretization on the coarse grid with the same preconditioning matrix 
as used on the fine grid gives, since Az, = 2Az, 


7 Az? 


1 
С-—»В(Мь:1,—2,1) = FG B(Me : 1,—2,1) = 7B(Me:1,-2,1) (10.40) 
Te 


? 


which is precisely the matrix appearing in Eq. 10.37. Thus we see that the process is 
recursive. The problem to be solved on the coarse grid is the same as that solved on 
the fine grid. 

The remaining steps required to complete an entire multigrid process are relatively 
straightforward, but they vary depending on the problem and the user. The reduction 
can be, and usually is, carried to even coarser grids before returning to the finest level. 
However, in each case the appropriate permutation matrix and the interpolation 
approximation define both the down- and up-going paths. The details of finding 
optimum techniques are, obviously, quite important but they are not discussed here. 
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10.3 A Two-Grid Process 


We now describe a two-grid process for the linear problem Аф = Ў, which can be easily 
generalized to a process with an arbitrary number of grids due to the recursive nature 
of multigrid. Extension to nonlinear problems requires that both the solution and the 
residual be transferred to the coarse grid in a process known as full approximation 
storage multigrid. 


1. Perform nı iterations of the selected relaxation method on the fine grid, starting 
with Ф = ọn. Call the result ФО). This gives! 


oY = GMb, + (1 Gn) A? f (10.41) 
where 
С, = I + НГА} (10.42) 


and H; is defined as in Chapter 9 (e.g., Eq. 9.21). Next compute the residual based 
on QU: 


rD = Ago F 


AGO PAC GAP Se 
AG 6, = AGRA TE (10.43) 


2. Transfer (or restrict) 1? to the coarse grid: 
PO = үг (10.44) 


In our example in the preceding section, the restriction matrix is 


010 
Н |0 00 (10.45) 
000 


that is, the first three rows of the permutation matrix P in Eq. 10.11. This type of 
restriction is known as "simple injection." Some form of weighted restriction can also 
be used. 
3. Solve the problem А„»ё? = ў) on the coarse grid exactly? 

eO = AMO (10.46) 


1See problem 1 of Chapter 9. 
?Note that the coarse grid matrix denoted A» here was denoted A, in the preceding section. 
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Here А» can be formed by applying the discretization on the coarse grid. In the 
preceding example (eq. 10.40), А» = 1в(м, :1,—2,1). It is at this stage that the 
generalization to a multigrid procedure with more than two grids occurs. If this is the 
coarsest grid in the sequence, solve exactly. Otherwise, apply the two-grid process 
recursively. 
4. Transfer (or prolong) the error back to the fine grid and update the solution: 
ong = OY — Be?) (10.47) 


In our example, the prolongation matrix is 


1/2 0 0 
1 0 0 
1/2 1/2 0 
=| 0 1 0 (10.48) 
0 1/2 1/2 
0 0 1l 
0 0 1/2 


which follows from Eq. 10.15. 
Combining these steps, one obtains 
&a-U-nmasnmRajesé-u-rnasmajgmA-P- AC (10.49) 
Thus the basic iteration matrix is 
[1 — I, Az R2 A\G? (10.50) 
The eigenvalues of this matrix determine the convergence rate of the two-grid process. 
The basic iteration matrix for a three-grid process is found from Eq. 10.50 by 
replacing Аў! with (J — G2) A5, where 
G2 = [I — IZ AZ| R2 A |С? (10.51) 
In this expression ny is the number of relaxation steps on grid 2, 72 and R3 are the 


transfer operators between grids 2 and 3, and A3 is obtained by discretizing on grid 
3. Extension to four or more grids proceeds in similar fashion. 
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10.4 Problems 


1. Derive Eq. 10.51. 
2. Repeat problem 4 of Chapter 9 using a four-grid multigrid method together 
with 
(a) the Gauss-Seidel method, 
(b) the 3-step Richardson method derived in Section 9.5. 
Solve exactly on the coarsest grid. Plot the solution after the residual is reduced 
by 2, 3, and 4 orders of magnitude. Plot the logarithm of the L2-norm of the 


residual vs. the number of iterations. Determine the asymptotic convergence 
rate. Calculate the theoretical asymptotic convergence rate and compare. 


Chapter 11 


NUMERICAL DISSIPATION 


Up to this point, we have emphasized the second-order centered-difference approxima- 
tions to the spatial derivatives in our model equations. We have seen that a centered 
approximation to a first derivative is nondissipative, i.e., the eigenvalues of the as- 
sociated circulant matrix (with periodic boundary conditions) are pure imaginary. 
In processes governed by nonlinear equations, such as the Euler and Navier-Stokes 
equations, there can be a continual production of high-frequency components of the 
solution, leading, for example, to the production of shock waves. In a real phys- 
ical problem, the production of high frequencies is eventually limited by viscosity. 
However, when we solve the Euler equations numerically, we have neglected viscous 
effects. Thus the numerical approximation must contain some inherent dissipation to 
limit the production of high-frequency modes. Although numerical approximations 
to the Navier-Stokes equations contain dissipation through the viscous terms, this 
can be insufficient, especially at high Reynolds numbers, due to the limited grid res- 
olution which is practical. Therefore, unless the relevant length scales are resolved, 
some form of added numerical dissipation is required in the numerical solution of 
the Navier-Stokes equations as well. Since the addition of numerical dissipation is 
tantamount to intentionally introducing nonphysical behavior, it must be carefully 
controlled such that the error introduced is not excessive. In this Chapter, we discuss 
some different ways of adding numerical dissipation to the spatial derivatives in the 
linear convection equation and hyperbolic systems of PDE's. 
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11.1 One-Sided First-Derivative Space Differenc- 
ing 


We investigate the properties of one-sided spatial difference operators in the context 
of the biconvection model equation given by 


сша а (11.1) 


with periodic boundary conditions. Consider the following point operator for the 
spatial derivative term 


-a(à,u); = 51-0 + В)ш; 1 + 28и; + (1 — В)и;] 
= acl Cui + иу) + B(—uj-1 + 2и; т uj41)] (11.2) 


The second form shown divides the operator into an antisymmetric component (—u; 44 
u541)/2Ax and a symmetric component @(—и;—1 + 2u; — uj41)/2Ac. The antisym- 
metric component is the second-order centered difference operator. With 8 Æ 0, the 
operator is only first-order accurate. A backward difference operator is given by 3 = 1 
and a forward difference operator is given by @ = —1. 


For periodic boundary conditions the corresponding matrix operator is 


—a 
—@6б = zaz C! = В, 28, 1— B) 


The eigenvalues of this matrix are 


— 2 2 
Am = == {8 [1 - cos (=) +isin (=) } for ш= 0,1,...,М-1 


If а is positive, the forward difference operator (8 = —1) produces Re(Am) > 0, 
the centered difference operator (8 = 0) produces Re(A,,) = 0, and the backward 
difference operator produces Re(A,,) < 0. Hence the forward difference operator is 
inherently unstable while the centered and backward operators are inherently stable. 
If a is negative, the roles are reversed. When Re(A,,) = 0, the solution will either 
grow or decay with time. In either case, our choice of differencing scheme produces 
nonphysical behavior. We proceed next to show why this occurs. 
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11.2 The Modified Partial Differential Equation 


First carry out a Taylor series expansion of the terms in Eq. 11.2. We are lead to the 


expression 
ди o (и Az? (0*u 8Ах* (dtu 
GR) co Eos 12 Ae 


We see that the antisymmetric portion of the operator introduces odd derivative 
terms in the truncation error while the symmetric portion introduces even derivatives. 
Substituting this into Eq. 11.1 gives 





(92и); = IAT 





ðu ди а8Ахд?и алх? и a BA? Ou 

Oi “Oc 2 Oe 6 OD 24 Әй 
This is the partial differential equation we are really solving when we apply the 
approximation given by Eq. 11.2 to Eq. 11.1. Notice that Eq. 11.3 is consistent with 
Eq. 11.1, since the two equations are identical when Az — 0. However, when we use 
a computer to find a numerical solution of the problem, Az can be small but it is 
not zero. This means that each term in the expansion given by Eq. 11.3 is excited to 
some degree. We refer to Eq. 11.3 as the modified partial differential equation. We 
proceed next to investigate the implications of this concept. 

Consider the simple linear partial differential equation 











(11.3) 


ди ди " ru à: Pu " otu (11.4) 
—а—— Lv — + т—— : 

z 7923 Ox 

Choose periodic boundary conditions and impose an initial condition и = e’**. Under 

these conditions there is a wave-like solution to Eq. 11.4 of the form 


u(x, t) = eife trio 
provided r and s satisfy the condition 
r+is = —iak — vk? — iyi? + Tk 
ог 
r——kK(v—cTk) в=—к(а+ yk’) 


The solution is composed of both amplitude and phase terms. Thus 
—K?(v—TK?) eile (a &?)t] (11.5) 
_—— 

amplitude phase 


U=eE 
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It is important to notice that the amplitude of the solution depends only upon v and 
т, the coefficients of the even derivatives in Eq. 11.4, and the phase depends only on 
a and y, the coefficients of the odd derivatives. 


If the wave speed a is positive, the choice of a backward difference scheme (8 = 1) 
produces a modified PDE with v — r&? > 0 and hence the amplitude of the solution 
decays. This is tantamount to deliberately adding dissipation to the PDE. Under the 
same condition, the choice of a forward difference scheme (8 = —1) is equivalent to 
deliberately adding a destabilizing term to the PDE. 


By examining the term governing the phase of the solution in Eq. 11.5, we see 
that the speed of propagation is a+ yx”. Referring to the modified PDE, Eq. 11.3 
we have y = —aAz?/6. Therefore, the phase speed of the numerical solution is less 
than the actual phase speed. Furthermore, the numerical phase speed is dependent 
upon the wavenumber к. This we refer to as dispersion. 


Our purpose here is to investigate the properties of one-sided spatial differencing 
operators relative to centered difference operators. We have seen that the three- 
point centered difference approximation of the spatial derivative produces a modified 
PDE that has no dissipation (or amplification). One can easily show, by using the 
antisymmetry of the matrix difference operators, that the same is true for any cen- 
tered difference approximation of a first derivative. As a corollary, ату departure 
from antisymmetry in the matriz difference operator must introduce dissipation (or 
amplification) into the modified PDE. 


Note that the use of one-sided differencing schemes is not the only way to in- 
troduce dissipation. Any symmetric component in the spatial operator introduces 
dissipation (or amplification). Therefore, one could choose 8 = 1/2 in Eq. 11.2. The 
resulting spatial operator is not one-sided but it is dissipative. Biased schemes use 
more information on one side of the node than the other. For example, a third-order 
backward-biased scheme is given by 


1 


6Ax 


1 
= IA = 8uj1 + 8uj41 = Uj +42) 


+ (uj—a — 4uj-i + 6u; — 40 + 32) (11.6) 


(би); = (uj—2 = 6uj—ı + 3uj + 2u541) 


The antisymmetric component of this operator is the fourth-order centered difference 
operator. The symmetric component approximates Az?^u;;;,/12. Therefore, this 
operator produces fourth-order accuracy in phase with a third-order dissipative term. 
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11.3 The Lax-Wendroff Method 


In order to introduce numerical dissipation using one-sided differencing, backward 

differencing must be used if the wave speed is positive, and forward differencing must 

be used if the wave speed is negative. Next we consider a method which introduces 

dissipation independent of the sign of the wave speed, known as the Lax-Wendroff 

method. This explicit method differs conceptually from the methods considered pre- 

viously in which spatial differencing and time-marching are treated separately. 
Consider the following Taylor-series expansion in time: 


Ou 1, „д?и Р 
= = 11. 
и(ж, +h) uc hb oh ag t OQ) (11.7) 


First replace the time derivatives with space derivatives according to the PDE (in 
this case, the linear convection equation ди + ади = 0). Тһиѕ 


ди __ ди ди _ зди 
a 9p Әр” сд? 


(11.8) 
ing 


2 
1 ah 
ur) en). (и) = u$) + 2 (5 | (иу = 24% zi uA) (11.9) 


This is the Lax-Wendroff method applied to the linear convection equation. It is a 
fully-discrete finite-difference scheme. There is no intermediate semi-discrete stage. 
For periodic boundary conditions, the corresponding fully-discrete matrix operator 


9 1 | ah ah | ану] ah ah | " 
си. E meae) Ne “у | a 
The eigenvalues of this matrix are 


2 
2 h 2 
„=1- (ҖЕ) [1 оов (=) | - i sin (£2) for m=0,1,...,M-1 


is 


Ag Ax 


For |2*| < 1 all of the eigenvalues have modulus less than or equal to unity and hence 
the method is stable independent of the sign of a. The quantity | is known as the 
Courant (or CFL) number. It is equal to the ratio of the distance travelled by a wave 
in one time step to the mesh spacing. 
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The nature of the dissipative properties of the Lax- Wendroff scheme can be seen 

by examining the modified partial differential equation, which is given by 

ðu ðu a u ah otu 

— the = —-—(Ах* aon) — — (Ar? — а2Һ?)— +... 

Ot Ox 6 | тз 8 ( ) Эт“ 
This is derived by substituting Taylor series expansions for all terms in Eq. 11.9 and 
converting the time derivatives to space derivatives using Eq. 11.8. The two leading 
error terms appear on the right side of the equation. Recall that the odd derivatives on 
the right side lead to unwanted dispersion and the even derivatives lead to dissipation 
(or amplification, depending on the sign). Therefore, the leading error term in the 
Lax-Wendroff method is dispersive and proportional to 








а Pu a^r? Pu 
EE ^ 25. «4 252 HOS EHE, t= C? REVO 
i а Jays 6 ( n) 5g 
The dissipative term is proportional to 
a?h otu a? hAz? otu 
СЕ Т NN je Е 
8 Е ag 8 ( DE 


This term has the appropriate sign and hence the scheme is truly dissipative as long 
as C, < 1. 

A closely related method is that of MacCormack. Recall MacCormack's time- 
marching method, presented in Chapter 6: 


фы 1 
loe = Un + hu, 





1 К Я 
иһ = эп + sid А0 i] (11.10) 


If we use first-order backward differencing in the first stage and first-order forward 
differencing in the second stage,! a dissipative second-order method is obtained. For 
the linear convection equation, this approach leads to 





~(n+1 n ah n n 
т у за A ) _ uf) 
n+ 1 n ~(n ah ~(n ~(n 
(P = Dla + ШЕЕ capt (11.11) 


which can be shown to be identical to the Lax-Wendroff method. Hence MacCor- 
mack's method has the same dissipative and dispersive properties as the Lax- Wendroff 
method. The two methods differ when applied to nonlinear hyperbolic systems, how- 
ever. 


!Or vice-versa; for nonlinear problems, these should be applied alternately. 
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11.4 Upwind Schemes 


In Section 11.1, we saw that numerical dissipation can be introduced in the spatial 
difference operator using one-sided difference schemes or, more generally, by adding 
a symmetric component to the spatial operator. With this approach, the direction 
of the one-sided operator (i.e., whether it is a forward or a backward difference) 
or the sign of the symmetric component depends on the sign of the wave speed. 
When a Ayperbolic system of equations is being solved, the wave speeds can be both 
positive and negative. For example, the eigenvalues of the flux Jacobian for the one- 
dimensional Euler equations are и, и -- a, и — a. When the flow is subsonic, these are 
of mixed sign. In order to apply one-sided differencing schemes to such systems, some 
form of splitting is required. This is avoided in the Lax-Wendroff scheme. However, 
as a result of their superior flexibility, schemes in which the numerical dissipation 
is introduced in the spatial operator are generally preferred over the Lax-Wendroff 


approach. 
Consider again the linear convection equation: 
ди ди 
—— nes] 11.12 
a “д hy) 


where we do not make any assumptions as to the sign of a. We can rewrite Eq. 11.12 
as 








Ou Е ds alal 

AES —-=(0 : SE Les 

a t (a* +a )5= Y a 5 
If а > 0, then at = а > 0 апа аг = 0. Alternatively, if a < 0, then at = 0 and 
a- =a < 0. Now for еа? (> 0) term we can safely backward difference and for the 


a` (< 0) term forward difference. This is the basic concept behind upwind methods, 
that is, some decomposition or splitting of the fluxes into terms which have positive 
and negative characteristic speeds so that appropriate differencing schemes can be 
chosen. In the next two sections, we present two splitting techniques commonly used 
with upwind methods. These are by no means unique. 

The above approach to obtaining a stable discretization independent of the sign 
of a can be written in a different, but entirely equivalent, manner. From Eq. 11.2, we 
see that a stable discretization is obtained with 8 = 1 if a > 0 and with 8 = —1 if 
а < 0. This is achieved by the following point operator: 


-a(8.u); = з= 


[a(—uj-1 + uj+1)  lal(—uj-1 + 2u; — uj+1)] (11.13) 
This approach is extended to systems of equations in Section 11.5. 

In this section, we present the basic ideas of flux-vector and flux-difference splitting. 
For more subtle aspects of implementation and application of such techniques to 


210 CHAPTER 11. NUMERICAL DISSIPATION 


nonlinear hyperbolic systems such as the Euler equations, the reader is referred to 
the literature on this subject. 


11.4.1 Flux-Vector Splitting 


Recall from Section 2.5 that a linear, constant-coefficient, hyperbolic system of partial 
differential equations given by 


Ou Of ди Ou 
ar ar Br ^B = 0 (11.14) 


can be decoupled into characteristic equations of the form 


дш; Кт 
дї * Or 





=0 (11.15) 


where the wave speeds, A;, are the eigenvalues of the Jacobian matrix, A, and the 
w;'s are the characteristic variables. In order to apply a one-sided (or biased) spatial 
differencing scheme, we need to apply a backward difference if the wave speed, A;, is 
positive, and a forward difference if the wave speed is negative. To accomplish this, 
let us split the matrix of eigenvalues, A, into two components such that 


A Sa АТ (11.16) 
where 
А+|А A — |^ 
m mE ez = (11.17) 


With these definitions, At contains the positive eigenvalues and A^ contains the neg- 
ative eigenvalues. We can now rewrite the system in terms of characteristic variables 
as 


—— — = — +AT— +A —=0 11.18 

On Ot Os ^ Ot ne) 
The spatial terms have been split into two components according to the sign of the 
wave speeds. We can use backward differencing for the At owe term and forward 
differencing for the A $e term. Premultiplying by X and inserting the product 
XTX in the spatial terms gives 


Xw OXA*'X Xw n OXA XXw 


"ae Po Sm 2c D 
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With the definitions? 

ARSON SATS (11.20) 
and recalling that и = Xw, we obtain 


Ou OA*u дА-и 








a =0 11.21 
a ðr ^ Ox Ue 
Finally the split flux vectors are defined as 
qeu f Au (11.22) 
and we can write 
Ou Of* Of- 
I- Ls АС lw 11.23 
дї Ox Ox ( ) 


In the linear case, the definition of the split fluxes follows directly from the defini- 
tion of the flux, f — Au. For the Euler equations, f is also equal to Au as a result of 
their homogeneous property, as discussed in Appendix C. Note that 


f=ff+f (11.24) 


Thus by applying backward differences to the f* term and forward differences to the 
fT term, we are in effect solving the characteristic equations in the desired manner. 
This approach is known as flux-vector splitting. 
When an implicit time-marching method is used, the Jacobians of the split flux 
vectors are required. In the nonlinear case, 
ERST EE PTS 
a 74 } a А (11.25) 


Therefore, опе must find and use the new Jacobians given by 


oft zu. of 
HIE A eR 11.26 
дм” ди ) 


For the Euler equations, A++ has eigenvalues which are all positive, and ATT has all 
negative eigenvalues. 


?With these definitions А has all positive eigenvalues, and A^ has all negative eigenvalues. 
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11.4.2 Flux-Difference Splitting 


Another approach, more suited to finite-volume methods, is known as flux-difference 
splitting. In a finite-volume method, the fluxes must be evaluated at cell bound- 
aries. We again begin with the diagonalized form of the linear, constant-coefficient, 
hyperbolic system of equations 


C E RO en 11.27 

дї Ox ( ) 
The flux vector associated with this form is g = Aw. Now, as in Chapter 5, we 
consider the numerical flux at the interface between nodes j and j + 1, Gj+1/2, as a 
function of the states to the left and right of the interface, wr, and tg, respectively. 
The centered approximation to gj+1/2, which is nondissipative, is given by 


А 1 
ĝj+1/2 = g (g(wr) + g(wr)) (11.28) 
In order to obtain a one-sided upwind approximation, we require 


А (о) г, if А; > 0 


(#)у+1/2 = | оло (11.29) 


where the subscript ? indicates individual components of ш and g. This is achieved 
with 


(наз = s (ш) + Gn] + о) — Qus) (11.30) 
j 1 1 
93+1/2 = ^ (wr, + шк) + |А (wr, — we) (11.31) 


Now, as in Eq. 11.19, we premultiply by X to return to the original variables and 
insert the product X^ X after А and |A] to obtain 


А 1 1 _ 
Хауз = JAX 1X (wr + wp) + 3XIAIX 1X (wr — wg) (11.32) 
and thus 
2 1 1 
Jas z Vr fg) + 5|A| (ur — ив) (11.33) 


where 


|A| = X|A|X7? (11.34) 
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and we have also used the relations f = Xg, u = Xw, and A= XAX-!. 

In the linear, constant-coefficient case, this leads to an upwind operator which is 
identical to that obtained using flux-vector splitting. However, in the nonlinear case, 
there is some ambiguity regarding the definition of |A| at the cell interface j + 1/2. 
In order to resolve this, consider a situation in which the eigenvalues of A are all of 
the same sign. In this case, we would like our definition of fin to satisfy 


fr if all A's > 0 


Ју = | Ín if all М; < 0 (11.35) 


giving pure upwinding. If the eigenvalues of A are all positive, |A| = A; if they are 
all negative, |A| = — А. Hence satisfaction of Eq. 11.35 is obtained by the definition 


А 1 1 
fins = 5 Ол. + fr) + Азу! (ur — un) (11.36) 
if Aj 4/5 satisfies 
ft — fr = Азро (ur — ив) (11.37) 


For the Euler equations for a perfect gas, Eq. 11.37 is satisfied by the flux Jacobian 
evaluated at the Roe-average state given by 


V PLUI + ./PRUR ( ) 

1/2 = VPL VPR 11 38 

H; 1/2 = a E (11 39) 
7+ / PL VPR А, 


where и and Н = (e + p)/p are the velocity and the total enthalpy per unit mass, 
respectively.? 


Uj 





11.5 Artificial Dissipation 


We have seen that numerical dissipation can be introduced by using one-sided dif- 
ferencing schemes together with some form of flux splitting. We have also seen that 
such dissipation can be introduced by adding a symmetric component to an antisym- 
metric (dissipation-free) operator. Thus we can generalize the concept of upwinding 
to include any scheme in which the symmetric portion of the operator is treated in 
such a manner as to be truly dissipative. 


3Note that the flux Jacobian can be written in terms of u and H only; see problem 6 at the end 
of this chapter. 
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For example, let 


_ Wjna c ua sy _ cuin t 2u; — uj 

(ти), = TEL MAL, (gni), = Se 

Applying à, = 6% + 0% to the spatial derivative in Eq. 11.15 is stable if A; > 0 and 

unstable if А; < 0. Similarly, applying ôs = 02 — ó; is stable if А; < 0 and unstable if 
A; » 0. The appropriate implementation is thus 


(11.40) 


Мб = Ai; + А; [9 (11.41) 


Extension to a hyperbolic system by applying the above approach to the characteristic 
variables, as in the previous two sections, gives 


&, (Au) = ô (Au) + 68 (|A[u) (11.42) 
Or 
Saf = ôf + б°(|А|и) (11.43) 


where |A| is defined in Eq. 11.34. The second spatial term is known as artificial 
dissipation. It is also sometimes referred to as artificial diffusion or artificial viscosity. 
With appropriate choices of 6° and 0°, this approach can be related to the upwind 
approach. This is particularly evident from a comparison of Eqs. 11.36 and 11.43. 

It is common to use the following operator for 57 


€ 
(92и), = Az 5-2 — 4uj-1 + би; — 4и; + изә) (11.44) 


where є is a problem-dependent coefficient. This symmetric operator approximates 
€Ax?u,,,, and thus introduces a third-order dissipative term. With an appropriate 
value of e, this often provides sufficient damping of high frequency modes without 
greatly affecting the low frequency modes. For details of how this can be implemented 
for nonlinear hyperbolic systems, the reader should consult the literature. A more 
complicated treatment of the numerical dissipation is also required near shock waves 
and other discontinuities, but is beyond the scope of this book. 


11.6 Problems 


1. A second-order backward difference approximation to a 1st derivative is given 
as a point operator by 


(беи), = — 4шу—1 + Зи;) 


ed 
———(u;. 
2Ах 2—2 
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(a) Express this operator in banded matrix form (for periodic boundary condi- 
tions), then derive the symmetric and skew-symmetric matrices that have 
the matrix operator as their sum. (See Appendix A.3 to see how to con- 
struct the symmetric and skew-symmetric components of a matrix.) 


(b) Using a Taylor table, find the derivative which is approximated by the 
corresponding symmetric and skew-symmetric operators and the leading 
error term for each. 


2. Find the modified wavenumber for the first-order backward difference operator. 
Plot the real and imaginary parts of &* Ax vs. kâz for 0 < &Az < m. Using 
Fourier analysis as in Section 6.6.2, find |o| for the combination of this spatial 
operator with 4th-order Runge-Kutta time marching at a Courant number of 
unity and plot vs. kAzx for 0 < KAT < m. 


3. Find the modified wavenumber for the operator given in Eq. 11.6. Plot the real 
and imaginary parts of &*Az vs. &Az for 0 € &Az < л. Using Fourier analysis 
as in Section 6.6.2, find |o| for the combination of this spatial operator with 
4th-order Runge-Kutta time marching at a Courant number of unity and plot 
vs. KAx for 0 € KAT < m. 


4. Consider the spatial operator obtained by combining second-order centered dif- 
ferences with the symmetric operator given in Eq. 11.44. Find the modified 
wavenumber for this operator with є = 0, 1/12, 1/24, and 1/48. Plot the real 
and imaginary parts of &*Az vs. &Az for 0 < &Az < л. Using Fourier analysis 
as in Section 6.6.2, find |o| for the combination of this spatial operator with 
4th-order Runge-Kutta time marching at a Courant number of unity and plot 
vs. KAx for 0 € KAT < m. 


5. Consider the hyperbolic system derived in problem 8 of Chapter 2. Find the 
matrix |A|. Form the plus-minus split flux vectors as in Section 11.4.1. 


6. Show that the flux Jacobian for the 1-D Euler equations can be written in terms 
of u and H. Show that the use of the Roe average state given in Eqs. 11.38 and 
11.39 leads to satisfaction of Eq. 11.37. 
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Chapter 12 


SPLIT AND FACTORED FORMS 


In the next two chapters, we present and analyze split and factored algorithms. This 
gives the reader a feel for some of the modifications which can be made to the basic 
algorithms in order to obtain efficient solvers for practical multidimensional applica- 
tions, and a means for analyzing such modified forms. 


12.1 The Concept 


Factored forms of numerical operators are used extensively in constructing and ap- 
plying numerical methods to problems in fluid mechanics. They are the basis for a 
wide variety of methods variously known by the labels “hybrid”, “time split”, and 
“fractional step”. Factored forms are especially useful for the derivation of practical 
algorithms that use implicit methods. When we approach numerical analysis in the 
light of matrix derivative operators, the concept of factoring is quite simple to present 
and grasp. Let us start with the following observations: 


1. Matrices can be split in quite arbitrary ways. 


2. Advancing to the next time level always requires some reference to a previous 
one. 


3. Time marching methods are valid only to some order of accuracy in the step 
size, h. 


Now recall the generic ODE’s produced by the semi-discrete approach 


du "E 
Gq = Aunt (12.1) 
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and consider the above observations. From observation 1 (arbitrary splitting of A) : 


dü "E 
2 = [A1 + Aqu — f (12.2) 


where A = [Ау + Ag] but А; and Ag are not unique. For the time march let us choose 
the simple, first-order,! explicit Euler method. Then, from observation 2 (new data 
Un+1 in terms of old tp): 
Una = [1 +hA: hAs]us — hf + O(h?) (12.3) 
or its equivalent 
ner = [1 +ҺА[1 + АА] — АА Аза, — Af + O(h?) 

Finally, from observation 3 (allowing us to drop higher order terms ҺА Actin): 

usi m [I +hAi][ I + hAglu, — hf + O(h?) (12.4) 


Notice that Eqs. 12.3 and 12.4 have the same formal order of accuracy and, in 
this sense, neither one is to be preferred over the other. However, their numerical 
stability can be quite different, and techniques to carry out their numerical evaluation 
can have arithmetic operation counts that vary by orders of magnitude. Both of these 
considerations are investigated later. Here we seek only to apply to some simple cases 
the concept of factoring. 


12.2 Factoring Physical Representations — Time 
Splitting 


Suppose we have a PDE that represents both the processes of convection and dissi- 
pation. The semi-discrete approach to its solution might be put in the form 


du » "HEN 
— = Аи + Aqu + (be) (12.5) 


where A, and Ag are matrices representing the convection and dissipation terms, 
respectively; and their sum forms the A matrix we have considered in the previous 
sections. Choose again the explicit Euler time march so that 


unii = [I +hAg + hA|tin + h(be) + O(h?) (12.6) 


!Second-order time-marching methods are considered later. 
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Now consider the factored form 


ә 


паа = [Т +ҺАД({1 +hAciin + h(be)) 

[I +hAg + hA us + h(be) + h?Ag(Actin + (be)) +O?) (12.7) 
Va Sure ED ET ———— 

Original Unfactored Terms Higher Order Terms 


and we see that Eq. 12.7 and the original unfactored form Eq. 12.6 have identical 
orders of accuracy in the time approximation. Therefore, on this basis, their selection 
is arbitrary. In practical applications? equations such as 12.7 are often applied in a 
predictor-corrector sequence. In this case one could write 


йы = [Z hAqu, + (bc) 


Un+1 = [I + ҺАа|йл+1 (12.8) 








Factoring can also be useful to form split combinations of implicit and explicit 
techniques. For example, another way to approximate Eq. 12.6 with the same order 
of accuracy is given by the expression 


ә 


usa = [I ҺА (1 +hAcltin + h(be)) 
[I +hAa + ҺА Ја, + h(bc) +O(h?) (12.9) 


Original Unfactored Terms 


where in this approximation we have used the fact that 
[I — hAq| = I c hAq + АА e... 


if h- ||Aal| < 1, where ||A4]| is some norm of [Aq]. This time a predictor-corrector 
interpretation leads to the sequence 


йаа = [I +ҺАДш+Һ(бе) 
[I —hAqus = бы (12.10) 





The convection operator is applied explicitly, as before, but the diffusion operator is 
now implicit, requiring a tridiagonal solver if the diffusion term is central differenced. 
Since numerical stiffness is generally much more severe for the diffusion process, this 
factored form would appear to be superior to that provided by Eq. 12.8. However, 
the important aspect of stability has yet to be discussed. 


?We do not suggest that this particular method is suitable for use. We have yet to determine its 
stability, and a first-order time-march method is usually unsatisfactory. 
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We should mention here that Eq. 12.9 can be derived for a different point of view 
by writing Eq. 12.6 in the form 


mn = Acus + Ализа + (bc) + O(h?) 
Then _ 
[I – ҺА и = [1 + hA,]u, + h(bc) 


which is identical to Eq. 12.10. 


12.3 Factoring Space Matrix Operators in 2-D 


12.3.1 Mesh Indexing Convention 


Factoring is widely used in codes designed for the numerical solution of equations 
governing unsteady two- and three-dimensional flows. Let us study the basic concept 
of factoring by inspecting its use on the linear 2-D scalar PDE that models diffusion: 

ðu u ди 


Oe OP 
We begin by reducing this PDE to a coupled set of ODE’s by differencing the space 
derivatives and inspecting the resulting matrix operator. 

A clear description of a matrix finite-difference operator in 2- and 3-D requires some 
reference to a mesh. We choose the 3 x 4 point mesh? shown in the Sketch 12.12. 
In this example М,, the number of (interior) x points, is 4 and M}, the number of 
(interior) y points is З. The numbers 11, 12, --- , 43 represent the location in the 
mesh of the dependent variable bearing that index. Thus изә represents the value of 
u at j = 3 and k = 2. 


(12.11) 


(12.12) 





Mesh indexing in 2-D. 


3This could also be called a 5 x 6 point mesh if the boundary points (labeled © in the sketch) 
were included, but in these notes we describe the size of a mesh by the number of interior points. 
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12.3.2 Data Bases and Space Vectors 


The dimensioned array in a computer code that allots the storage locations of the 
dependent variable(s) is referred to as a data-base. There are many ways to lay out 
a data-base. Of these, we consider only two: (1), consecutively along rows that are 
themselves consecutive from k = 1 to M,, and (2), consecutively along columns that 
are consecutive from 7 = 1 to M,. We refer to each row or column group as a 
space vector (they represent data along lines that are continuous in space) and label 
their sum with the symbol U. In particular, (1) and (2) above are referred to as z- 
vectors and y-vectors, respectively. The symbol U by itself is not enough to identify 
the structure of the data-base and is used only when the structure is immaterial or 
understood. 

To be specific about the structure, we label a data-base composed of x-vectors with 
U™) , and one composed of y-vectors with U(?). Examples of the order of indexing 
for these space vectors are given in Eq. 12.16 part a and b. 


12.3.3 Data Base Permutations 


The two vectors (arrays) are related by a permutation matrix P such that 


where 
Py; = P P P 
Now consider the structure of a matrix finite-difference operator representing 3- 
point central-differencing schemes for both space derivatives in two dimensions. When 
the matrix is multiplying a space vector U, the usual (but ambiguous) representation 
is given by А-у. In this notation the ODE form of Eq. 12.11 can be written * 


dU > 

If it is important to be specific about the data-base structure, we use the notation 
A or А» depending on the data-base chosen for the U it multiplies. Examples 
are in Eq. 12.16 part a and b. Notice that the matrices are not the same although 
they represent the same derivative operation. Their structures are similar, however, 


and they are related by the same permutation matrix that relates U to U™. Thus 
AG, = Р: АЧ Pye (12.15) 


‘Notice that А, ұу and U, which are notations used in the special case of space vectors, are 
subsets of А and и, used in the previous sections. 
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a:Elements in 2-dimensional, central-difference, matrix 
operator, Ау, for 3x4 mesh shown in Sketch 12.12. 
Data base composed of М, x-vectors stored in D. 
Entries for ж > x, for y > o, for both — e. 


(12.16) 


© 
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8 
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b: Elements in 2-dimensional, central-difference, matrix 
operator, Ауу, for 3x4 mesh shown in Sketch 12.12. 
Data base composed of M, y-vectors stored in U™). 

Entries for x — x, for y > o, for both — e. 
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12.3.4 Space Splitting and Factoring 


We are now prepared to discuss splitting in two dimensions. It should be clear that 
the matrix Аб), can be split into two matrices such that 


AC), = AG + AC (12.17) 
where A(? and AQ are shown in Eq. 12.22. Similarily 
AY), = AQ + AW (12.18) 


where the split matrices are shown in Eq. 12.23. 
The permutation relation also holds for the split matrices so 


and 
ДО) = Pry A0 P 


x 


The splittings in Eqs. 12.17 and 12.18 can be combined with factoring in the 
manner described in Section 12.2. As an example (first-order in time), applying the 
implicit Euler method to Eq. 12.14 gives 


Ut, = US) + АА + AC UC, + h(be) 
or 
[Т — nA(? — nA(9 ut, = UO + h(be) + O(n?) (12.19) 
As in Section 12.2, we retain the same first order accuracy with the alternative 


I — hAO|[r — nA U® 2 [7® + h(be) + O(h? 12.20 
£ y + n 


Write this in predictor-corrector form and permute the data base of the second row. 
There results 


-hapO = UP + hbe) 
0760) (12.21) 


кз 
| 
> 
D> 
& 
кең 
+© 
us 
| 
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The splitting of AC), . 
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The splitting of Ao. 
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12.4 Second-Order Factored Implicit Methods 


Second-order accuracy in time can be maintained in a certain factored implicit meth- 
ods. For example, apply the trapezoidal method to Eq. 12.14 where the derivative 
operators have been split as in Eq. 12.17 or 12.18. Let the data base be immaterial 
and the (bc) be time invariant. There results 


1 1 1 4 2 
[ - Sh, — 5h Use [ + 5А» + shAy| Un + h(be) + OQ) (12.24) 


Factor both sides giving 





1 1 1 | 
| |1 = 5А» |1 - shay - Fh? As Ay Ша 
1 1 { В 
= | 1 + hA, [ a sh Ay| - ThA, Ay] Un + Ме) OQ) (12.25) 





Then notice that the combination Ір [А А, (Сг — Un) is proportional to АЗ since 
the leading term in the expansion of (U,4; — Un) is proportional to h. Therefore, we 
can write 


1 1 
|1 - уһА, 1 AL 


2 
and both the factored and unfactored form of the trapezoidal method are second-order 
accurate in the time march. 

An alternative form of this kind of factorization is the classical ADI (alternating 
direction implicit) method? usually written 


1 1 E 
Uns = |1 n EM [ + shay] Un + h(be) + O(h?) (12.26) 








U 


Ї 1 1 
1— —ҺА„ I+ hA D sep: 
| 2 | т 2 ‚| Сат 2 





1 E TAE, 
a= нА) D 2 [ + sha | б + Sh Pass + O(h?) (12.27) 





For idealized commuting systems the methods given by Eqs. 12.26 and 12.27 differ 
only in their evaluation of a time-dependent forcing term. 


12.5 Importance of Factored Forms in 2 and 3 Di- 
mensions 


When the time-march equations are stiff and implicit methods are required to permit 
reasonably large time steps, the use of factored forms becomes a very valuable tool 


5A form of the Douglas or Peaceman-Rachford methods. 
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for realistic problems. Consider, for example, the problem of computing the time 
advance in the unfactored form of the trapezoidal method given by Eq. 12.24 
С == 


1 1 А 
I ћу rg БЛА) U, + h(be) 











Forming the right hand side poses no problem, but finding „1 requires the solution 
of a sparse, but very large, set of coupled simultaneous equations having the matrix 
form shown in Eq. 12.16 part a and b. Furthermore, in real cases involving the Euler 
or Navier-Stokes equations, each symbol (o, x, e) represents a 4 x 4 block matrix with 
entries that depend on the pressure, density and velocity field. Suppose we were to 
solve the equations directly. The forward sweep of a simple Gaussian elimination #1156 
all of the 4 x 4 blocks between the main and outermost diagonal! (e.g. between e 
and o in Eq. 12.16 part b.). This must be stored in computer memory to be used to 
find the final solution in the backward sweep. If N, represents the order of the small 
block matrix (4 in the 2-D Euler case), the approximate memory requirement is 


(Ne x My) - (Ne x M,)- Ma 


floating point words. Here it is assumed that M, < M;. If M, > M;, M, and М, 
would be interchanged. А moderate mesh of 60 x 200 points would require over 11 
million words to find the solution. Actually current computer power is able to cope 
rather easily with storage requirements of this order of magnitude. With computing 
speeds of over one gigaflop,® direct solvers may become useful for finding steady-state 
solutions of practical problems in two dimensions. However, a three-dimensional 
solver would require a memory of approximatly 


N2-M;-M2-M, 


words and, for well resolved flow fields, this probably exceeds memory availability for 
some time to come. 

On the other hand, consider computing a solution using the factored implicit equa- 
tion 12.25. Again computing the right hand side poses no problem. Accumulate the 
result of such a computation in the array (RH S). One can then write the remaining 
terms in the two-step predictor-corrector form 


1 E 
|r- sna ŬO = (RHS)® 





1 d 
[ 2 shal? | 00), = pe (12.28) 

®For matrices as small as those shown there are many gaps in this “fill”, but for meshes of 
practical size the fill is mostly dense. 

"The lower band is also computed but does not have to be saved unless the solution is to be 
repeated for another vector. 

8One billion floating-point operations per second. 
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which has the same appearance as Eq. 12.21 but is second-order time accurate. The 
first step would be solved using M, uncoupled block tridiagonal solvers?. Inspecting 
the top of Eq. 12.22, we see that this is equivalent to solving M, one-dimensional 
problems, each with M, blocks of order Ne. The temporary solution U? would then 
be permuted to U(? and an inspection of the bottom of Eq. 12.23 shows that the final 
step consists of solving M, one-dimensional implicit problems each with dimension 
M,. 


12.6 The Delta Form 


Clearly many ways can be devised to split the matrices and generate factored forms. 
One way that is especially useful, for ensuring a correct steady-state solution in a 
converged time-march, is referred to as the “delta form" and we develop it next. 

Consider the unfactored form of the trapezoidal method given by Eq. 12.24, and 
let the (bc) be time invariant: 


1 1 1 1 : 
I — ShA; 5А Usi = |1 + п, + shay] Un + h(be) + O(n?) 





From both sides subtract i 1 
I - sha, — 5h U, 


leaving the equality unchanged. Then, using the standard definition of the difference 
operator A, 
AU, = Until — Un 


one finds 
1 1 5 
I= 5А. — 5h, AU, = МА 0, + (б©)] + OR?) (12.29) 


Notice that the right side of this equation is the product of h and a term that is 
identical to the right side of Eq. 12.14, our original ODE. Thus, if Eq. 12.29 converges, 
it is guaranteed to converge to the correct steady-state solution of the ODE. Now we 
can factor Eq. 12.29 and maintain O(h?) accuracy. We arrive at the expression 


1 1 > 
I - ShÀ, 1 = shy] AU, = h[AssyUn + (6c)] + O(h’) (12.30) 





This is the delta form of a factored, 2nd-order, 2-D equation. 


ЗА block tridiagonal solver is similar to a scalar solver except that small block matrix operations 
replace the scalar ones, and matrix multiplications do not commute. 
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'The point at which the factoring is made may not affect the order of time-accuracy, 
but it can have a profound effect on the stability and convergence properties of a 
method. For example, the unfactored form of a first-order method derived from the 
implicit Euler time march is given by Eq. 12.19, and if it is immediately factored, 
the factored form is presented in Eq. 12.20. On the other hand, the delta form of the 
unfactored Eq. 12.19 is 


[I — hA; — hAJAU, = h| A, 4U, + (bc) 


and its factored form becomes!? 


[I — hA;]I — hA,JAU, = h| Az uU, + (60)] (12.31) 


In spite of the similarities in derivation, we will see in the next chapter that the 
convergence properties of Eq. 12.20 and Eq. 12.31 are vastly different. 


12.7 Problems 
1. Consider the 1-D heat equation: 


ди Oru 
p <r< 
gg = 


Let u(0,t) = 0 and u(9, t) = 0, so that we can simplify the boundary conditions. 
Assume that second order central differencing is used, i.e., 


1 
(дьи) = Aca (uji — 2и) + шучы) 
The uniform grid has Ax = 1 and 8 interior points. 


(a) Space vector definition 
i. What is the space vector for the natural ordering (monotonically in- 
creasing in index), u(? ? Only include the interior points. 


ii. If we reorder the points with the odd points first and then the even 
points, write the space vector, и?) ? 


iii. Write down the permutation matrices, ( Pi», P5). 


10Notice that the only difference between the O(h?) method given by Eq. 12.30 and the O(h) 
method given by Eq. 12.31 is the appearance of the factor 1 оп the left side of the O(h?) method. 
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iv. The generic ODE representing the discrete form of the heat equation 


1S 





Write down the matrix Ау. (Note f = 0, due to the boundary condi- 
tions) Next find the matrix A» such that 





du? 
= Аш 
di s 
Note that As can be written as 
D | UT 
A» = 
U ID 








Define D and U. 


. Applying implicit Euler time marching, write the delta form of the 


implicit algorithm. Comment on the form of the resulting implicit 
matrix operator. 


(b) System definition 
In problem 1а, we defined и), и), Ay, Ao, Pio, and P», which partition 
the odd points from the even points. We can put such a partitioning to 
use. First define extraction operators 


1000 0000 
0100 0000 
0010 0000 

zie. | 0: УЗО ооо 
0000 0000 EY 
0000 0000 * К 
0000 0000 
0000 0000 
0000 0000 
0000 0000 
0000 0000 NE 

ge = stp nr vg О ш. Шз 204 Е i 
0000 1000 miL 
0000 0100 : : 
0000 0010 
0000 000 1 
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which extract the odd even points from u) as follows: u = Iu?) and 
uy) — е0). 


i. Beginning with the ODE written in terms of и), define a splitting 
А» = A, + Ae, such that A, operates only on the odd terms, and A, 
operates only on the even terms. Write out the matrices A, and Ag. 
Also, write them in terms of D and U defined above. 

ii. Apply implicit Euler time marching to the split ODE. Write down the 
delta form of the algorithm and the factored delta form. Comment on 
the order of the error terms. 

iii. Examine the implicit operators for the factored delta form. Comment 
on their form. You should be able to argue that these are now trangu- 
lar matrices (a lower and an upper). Comment on the solution process 
this gives us relative to the direct inversion of the original system. 
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Chapter 13 


LINEAR ANALYSIS OF SPLIT 
AND FACTORED FORMS 


In Section 4.4 we introduced the concept of the representative equation, and used 
it in Chapter 7 to study the stability, accuracy, and convergence properties of time- 
marching schemes. The question is: Can we find a similar equation that will allow 
us to evaluate the stability and convergence properties of split and factored schemes? 
The answer is yes — for certain forms of linear model equations. 

The analysis in this chapter is useful for estimating the stability and steady-state 
properties of a wide variety of time-marching schemes that are variously referred 
to as time-split, fractional-step, hybrid, and (approximately) factored. When these 
methods are applied to practical problems, the results found from this analysis are 
neither necessary nor sufficient to guarantee stability. However, if the results indicate 
that a method has an instability, the method is probably not suitable for practical 
use. 


13.1 The Representative Equation for Circulant 
Operators 


Consider linear PDE's with coefficients that are fixed in both space and time and with 
boundary conditions that are periodic. We have seen that under these conditions 
a semi-discrete approach can lead to circulant matrix difference operators, and we 
discussed circulant eigensystems! in Section 4.3. In this and the following section 
we assume circulant systems and our analysis depends critically on the fact that all 
circulant matrices commute and have a common set of eigenvectors. 


!See also the discussion on Fourier stability analysis in Section 7.7. 
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Suppose, as a result of space differencing the PDE, we arrive at a set of ODE's 
that can be written 
du E E Eis 
em Ари + Appu — f(t) (13.1) 
where the subscript p denotes a circulant matrix. Since both matrices have the same 
set of eigenvectors, we can use the arguments made in Section 4.2.3 to uncouple the 
set and form the M set of independent equations 


wi = (А, + Ab) ил = gi(t) 


Win (Aa + 5) tbe gmlt) 


шм = (Aa + №) ушм — gmlt) (13.2) 
The analytic solution of the m’th line is 
Wm(t) = сле 9)" 4 P.S. 
Note that each А, pairs with one, and only one?, A, since they must share a common 


eigenvector. This suggests (see Section 4.4: 


'The representative equation for split, circulant systems is 


d 
= [Aa + Ab + Ae + ju + ае“ (13.3) 


dt 


where Aa + Àb + Àc +--+ аге the sum of the eigenvalues in A, , Ap, A, , ++- that 
share the same eigenvector. 





13.2 Example Analysis of Circulant Systems 


13.2.1 Stability Comparisons of Time-Split Methods 


Consider as an example the linear convection-diffusion equation: 


ди ди д?и 


?'This is to be contrasted to the developments found later in the analysis of 2-Р” equations. 
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If the space differencing takes the form 


du а E и E 
— = —— —B,(—1,0,1 —,; B, (1, —2,1 13:5 
the convection matrix operator and the diffusion matrix operator, can be represented 


by the eigenvalues А, and Ау, respectively, where (see Section 4.3.2): 


(us = Asin Onn 
4v . 5 Om 
(Ad) mn = CONSE sin? DE (13.6) 
In these equations 0, = 2тт/ М, т= 0, 1, ·--, M—1,5sothat 0 € 0, < 27. 


Using these values and the representative equation 13.4, we can analyze the stability 
of the two forms of simple time-splitting discussed in Section 12.2. In this section we 
refer to these as 


1. the explicit-implicit Euler method, Eq. 12.10. 


2. the explicit-explicit Euler method, Eq. 12.8. 


1. The Explicit-Implicit Method 
When applied to Eq. 13.4, the characteristic polynomial of this method is 
P(E) = (1— hA4)E — (1 + ҺА„) 


This leads to the principal с root 


h 
Dig and. 


E "T 
VY, m 
1+ 4 sin’ > 


where we have made use of Eq. 13.6 to quantify the eigenvalues. Now introduce the 
dimensionless numbers 





h 
C, = Св , Courant number 
Ax 
А 
Ra = ше. , mesh Reynolds number 
7 


and we can write for the absolute value of с 


\/1 + С? sin? Om 
л MS ЕЕ; (13.7) 
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Figure 13.1: Stability regions for two simple time-split methods. 


А simple numerical parametric study of Eq. 13.7 shows that the critical range of 
m for any combination of C, and Ra occurs when ĝm is near 0 (or 27). From this 
we find that the condition on C, and Ra that make |о| ~ 1 is 


C 2 
[1 + С? sin? e| = [ + ^R. sin? 5 
As є — 0 this gives the stability region 


2 
Cn < — 
Ra 


which is bounded by a hyperbola and shown in Fig. 13.1. 


2. The Explicit-Explicit Method 
An analysis similar to the one given above shows that this method produces 
С SO 
le| = y1 + C2 sin? 6,,|1-4— sin? —| , 0< 60, < 27 
Ra 2 


Again a simple numerical parametric study shows that this has two critical ranges 
of 0,,, one near 0, which yields the same result as in the previous example, and the 
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other near 180°, which produces the constraint that 
1 
Ch < 5 F^ for Ra <2 


The resulting stability boundary is also shown in Fig. 13.1. The totaly explicit, 
factored method has a much smaller region of stability when Ra is small, as we 
should have expected. 


13.2.2 Analysis of a Second-Order Time-Split Method 


Next let us analyze a more practical method that has been used in serious compu- 
tational analysis of turbulent flows. This method applies to a flow in which there is 
a combination of diffusion and periodic convection. The convection term is treated 
explicitly using the second-order Adams-Bashforth method. The diffusion term is 
integrated implicitly using the trapezoidal method. Our model equation is again the 
linear convection-diffusion equation 13.4 which we split in the fashion of Eq. 13.5. In 
order to evaluate the accuracy, as well as the stability, we include the forcing func- 
tion in the representative equation and study the effect of our hybrid, time-marching 
method on the equation 
uw = М№и + Aqu + ae 


First let us find expressions for the two polynomials, P(E) and Q(E). The char- 
acteristic polynomial follows from the application of the method to the homogeneous 
equation, thus 


1 1 
Until = Un + 5А (Зип = Un—ı) + g una + Un) 


This produces 


3 


P(E) = (1 — 3) E" —(1+ 5^ + ;h)E + 5! 


The form of the particular polynomial depends upon whether the forcing function is 
carried by the AB2 method or by the trapezoidal method. In the former case it is 


1 
Q(E) = 5h (3E — 1) (13.8) 
and in the latter 


Q(E) — PE + E) (13.9) 
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Accuracy 


From the characteristic polynomial we see that there are two c-roots and they are 
given by the equation 


3 1 3 I ү? 1 
1+ 5А + shat (1 + 5А + 58м) - 25A. (1 =: 58м) 
UEM Su Ne RNC САС) 


1 
old 
( 5) 


The principal о-гооё follows from the plus sign and one can show 





1 1 
ey 2 1+ (де + Aa)h + Z (Ae + А)? + 103 FAI — Аа = X) 


From this equation it is clear that pe = Ha. + Аа)? does not match the coefficient 
of h? in o1, so 


ету = O(h?) 
Using P(e"^) and Q(e"^) to evaluate ег, in Section 6.6.3, one can show 
er, = O(h?) 


using either Eq. 13.8 or Eq. 13.9. These results show that, for the model equation, 
the hybrid method retains the second-order accuracy of its individual components. 


Stability 


The stability of the method can be found from Eq. 13.10 by a parametric study of cn 
and Ra defined in Eq. 13.7. This was carried out in a manner similar to that used 
to find the stability boundary of the first-order explicit-implicit method in Section 
13.2.1. The results are plotted in Fig. 13.2. For values of Ra > 2 this second-order 
method has а much greater region of stability than the first-order explicit-implicit 
method given by Eq. 12.10 and shown in Fig. 13.1. 


13.3 The Representative Equation for Space-Split 
Operators 
Consider the 2-D model? equations 


Qu д?и Pu 


э Jz F БС (13.11) 


3The extension of the following to 3-D is simple and straightforward. 
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Figure 13.2: Stability regions for the second-order time-split method. 


and 


Qu Qu ди 

m = —=0 13:12 

LUN CT ста ete) 
Reduce either of these, by means of spatial differencing approximations, to the coupled 
set of ODE's: 


id = [A, + AJU + (bc) (13.13) 
for the space vector U. The form of the A; and A, matrices for three-point central 
differencing schemes are shown in Eqs. 12.22 and 12.23 for the 3 x 4 mesh shown 
in Sketch 12.12. Let us inspect the structure of these matrices closely to see how we 
can diagonalize | A; + A] in terms of the individual eigenvalues of the two matrices 
considered separately. 

First we write these matrices in the form 


B 
A@ = 


x 


B 








where B is a banded matrix of the form B(b_1, bo, b1). Now find the block eigenvector 
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matrix that diagonalizes B and use it to diagonalize A(?. Thus 


A B X А 


























where 


Аз 
А 
Notice that the matrix AQ is transparent to this transformation. That is, if we 
set X = diag( X) 


Dol del el bl 
ХБ. bl blo X-—bba del 6-1 
bi- 0-1 bal bI 














One now permutes the transformed system to the y-vector data-base using the per- 
mutation matrix defined by Eq. 12.13. There results 


P4: ХУА + AM] X - Pry = 


ESI [В _ | 
№-1 
Жы (13.14) 


àI 








where B is the banded tridiagonal matrix B(b_1, bo, b), see the bottom of Eq. 12.23. 
Next find the eigenvectors X that diagonalize the B blocks. Let B = diag(B) and 


X = diag( X) and form the second transformation 
А 
12) E ‚ i= 





This time, by the same argument as before, the first matrix on the right side of 
Eq. 13.14 is transparent to the transformation, so the final result is the complete 
diagonalization of the matrix Аз фу: 


pop cox PAL eese pes 


Abe 
Aic 
" 13.1 
Asl +A o) 


Mal d А 
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It is important to notice that: 


e The diagonal matrix on the right side of Eq. 13.15 contains every possible com- 
bination of the individual eigenvalues of B and B. 


Now we are ready to present the representative equation for two dimensional sys- 
tems. First reduce the PDE to ODE by some choice^ of space differencing. This 
results in a spatially split А matrix formed from the subsets 


where B and B are any two matrices that have linearly independent eigenvectors (this 
puts some constraints on the choice of differencing schemes). 





Although A; and A, do commute, this fact, by itself, does not ensure the prop- 
erty of “all possible combinations". To obtain the latter property the structure of 
the matrices is important. The block matrices B and B can be either circulant or 
noncirculant; in both cases we are led to the final result: 


'The 2-D representative equation for model linear systems is 


du 
"i [As + Alu + ае“ 


where A, and A, are any combination of eigenvalues from A, and A}, a and џ are 
(possibly complex) constants, and where A, and A, satisfy the conditions in 13.16. 





Often we are interested in finding the value of, and the convergence rate to, the 
steady-state solution of the representative equation. In that case we set u = 0 and 
use the simpler form 

du 


n [Ax + „и +a (13.17) 


which has the exact solution 





(13.18) 


^We have used 3-point central differencing in our example, but this choice was for convenience 
only, and its use is not necessary to arrive at Eq. 13.15. 
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13.4 Example Analysis of 2-D Model Equations 


In the following we analyze four different methods for finding a fixed, steady-state 
solution to the 2-D representative equation 13.16. In each case we examine 


1. The stability. 
2. The accuracy of the fixed, steady-state solution. 


3. The convergence rate to reach the steady-state. 


13.4.1 The Unfactored Implicit Euler Method 


Consider first this unfactored, first-order scheme which can then be used as a reference 
case for comparison with the various factored ones. The form of the method is given 
by Eq. 12.19, and when it is applied to the representative equation, we find 


(1 — h às — h ày Unyi = Un + ha 
from which 


PO) = ox уі 
Q(E) — h (13.19) 


giving the solution 


1 $ a 
Un = c| ———] —-——— 
] eg As RA, Aq + Ay 


Like its counterpart in the 1-D case, this method: 
1. Is unconditionally stable. 


2. Produces the exact (see Eq. 13.18) steady-state solution (of the ODE) for any 
h. 


3. Converges very rapidly to the steady-state when h is large. 


Unfortunately, however, use of this method for 2-D problems is generally impractical 
for reasons discussed in Section 12.5. 
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13.4.2 The Factored Nondelta Form of the Implicit Euler 
Method 


Now apply the factored Euler method given by Eq. 12.20 to the 2-D representative 
equation. There results 


(1 — h A;)(1 — h Ay)us = Un + ha 
from which 
P(E) = (1- hAQ(1—hX)E-—1 
О(Е) = h (13.20) 
giving the solution 
К 1 - e a 
(1- ЛА)(1- hry) Ag + А, ASA 
We see that this method: 


Un = 


1. Is unconditionally stable. 
2. Produces a steady state solution that depends on the choice of h. 


3. Converges rapidly to a steady-state for large h, but the converged solution is 
completely wrong. 


The method requires far less storage then the unfactored form. However, it is not 
very useful since its transient solution is only first-order accurate and, if one tries to 
take advantage of its rapid convergence rate, the converged value is meaningless. 


13.4.3 The Factored Delta Form of the Implicit Euler Method 
Next apply Eq. 12.31 to the 2-D representative equation. One finds 
[T SNe Sy) i — Un) = h(ArUn + Ayun + a) 
which reduces to 
(1 — hAz)(1— АА) = (1 + XA) Un + ha 
and this has the solution 


T =т=т. а 
S (1— hX)(1— h Aj) № + Ay 


'This method: 
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1. Is unconditionally stable. 
2. Produces the exact steady-state solution for any choice of Л. 


3. Converges very slowly to the steady-state solution for large values of h, since 
lo] — 1 as h — oo. 


Like the factored nondelta form, this method demands far less storage than the un- 
factored form, as discussed in Section 12.5. The correct steady solution is obtained, 
but convergence is not nearly as rapid as that of the unfactored form. 


13.4.4 The Factored Delta Form of the Trapezoidal Method 


Finally consider the delta form of a second-order time-accurate method. Apply Eq. 
12.30 to the representative equation and one finds 


1 1 
(1 2 j^) (1 T 58) lues sen м d ous и) 
which reduces to 
1 1 1 1 
(1 x zhe) (1 Е jh) Мыз (1 4 j^) (1 + 5) Un + ha 


and this has the solution 





This method: 


1. Is unconditionally stable. 
2. Produces the exact steady-state solution for any choice of h. 


3. Converges very slowly to the steady-state solution for large values of h, since 
lo] — 1 as h — oo. 


All of these properties are identical to those found for the factored delta form of the 
implicit Euler method. Since it is second order in time, it can be used when time 
accuracy is desired, and the factored delta form of the implicit Euler method can be 
used when a converged steady-state is all that is required.? A brief inspection of eqs. 
12.26 and 12.27 should be enough to convince the reader that the o’s produced by 
those methods are identical to the c produced by this method. 


5In practical codes, the value of h on the left side of the implicit equation is literally switched 
from h to Ah. 
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13.5 Example Analysis of the 3-D Model Equation 
The arguments in Section 13.3 generalize to three dimensions and, under the condi- 
tions given in 13.16 with an A(? included, the model 3-D cases? have the following 
representative equation (with и = 0): 
du 
P [Ac + А, + A;]u +a (13.21) 
Let us analyze a 2nd-order accurate, factored, delta form using this equation. First 
apply the trapezoidal method: 


Unt = Un + ЕВА, + Ау + Az)Ungi + (А + Ay + A;)us + 2a] 
Rearrange terms: 

[ = 5^0. a eel X) шя = 1 sh As ra eee el pedes 
Put this in delta form: 


1 
1- МА + Ay + X) Aus eh [Ug EA Aussi] 





Now factor the left side: 
1 1 1 
(1 j^) (1 = shay (1 = 57.) Atin = (АА А, а] (13.29) 


This preserves second order accuracy since the error terms 
1 
4 


are both O(h?). One can derive the characteristic polynomial for Eq. 13.22, find the 
с root, and write the solution either in the form 


1 
h? (Ag Ay + А.А, + АА) Аш and g MA 


1 1 1 n 
m 5h + Ay + М) + Zh? (МА, + МА, + AyAs) = ЗАЗА МА, | 


Un = C 
ü 1 1 1 
f — 5%» + Ay + Az) + т ody + МА, + АА,) — ААА, 


а 


Итеа о 
£ y 2, 


бЕаѕ. 13.11 and 13.12, each with an additional term. 
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or in the form 


1 1 1 1 n 

(1+ =.) (1+ srry) (1+ >һл„) — мм, : 

ЕЕЕ ЕЕН 
етет 


It is interesting to notice that a Taylor series expansion of Eq. 13.24 results in 


Un = С 


1 

ЖОРО Ce 5 О» dues (13.25) 
1 

+ grë DE (2y + Wz) + (2А + ЗАА + 24) + Ay + DAG E22 + AB] +: 


which verifies the second order accuracy of the factored form. Furthermore, clearly, 
if the method converges, it converges to the proper steady-state." 

With regards to stability, it follows from Eq. 13.23 that, if all the A's are real and 
negative, the method is stable for all h. This makes the method unconditionally stable 
for the 3-D diffusion model when it is centrally differenced in space. 

Now consider what happens when we apply this method to the biconvection model, 
the 3-D form of Eq. 13.12 with periodic boundary conditions. In this case, central 
differencing causes all of the A's to be imaginary with spectrums that include both 
positive and negative values. Remember that in our analysis we must consider every 
possible combination of these eigenvalues. First write the с root in Eq. 13.23 in the 
form 
ару 
~ l-ia-B+iy 
where a, 3 and y are real numbers that can have any sign. Now we can always find 
one combination of the A's for which a, and y are both positive. In that case since 
the absolute value of the product is the product of the absolute values 


с 


| = (1— 8)? + (0+7)? E 


(T= 8 (0—7)? 
and the method is unconditionally unstable for the model convection problem. 

From the above analysis one would come to the conclusion that the method rep- 
resented by Eq. 13.22 should not be used for the 3-D Euler equations. In practical 
cases, however, some form of dissipation is almost always added to methods that are 
used to solve the Euler equations and our experience to date is that, in the presence 
of this dissipation, the instability disclosed above is too weak to cause trouble. 


"However, we already knew this because we chose the delta form. 
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13.6 Problems 


1. Starting with the generic ODE, 


du 
ла A 
di ud f 


we can split А as follows: А = Ау + A2 + Аз + A4. Applying implicit Euler time 
marching gives 


Un+1 — Un 


5 = Ауџь р + Asus a + Азил: + Алчы + f 


(a) Write the factored delta form. What is the error term? 
(b) Instead of making all of the split terms implicit, leave two explicit: 


Un+1 — Un 


п = Ауиьці + Á2Un + Ази + Али + f 
Write the resulting factored delta form and define the error terms. 


(c) The scalar representative equation is 


du 

di (Ai + № + Ag + №М)и+а 

For the fully implicit scheme of problem 1a, find the exact solution to the 
resulting scalar difference equation and comment on the stability, conver- 
gence, and accuracy of the converged steady-state solution. 


(d) Repeat 1c for the explicit-implicit scheme of problem 1b. 
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Appendix A 


USEFUL RELATIONS AND 
DEFINITIONS FROM LINEAR 
ALGEBRA 


А basic understanding of the fundamentals of linear algebra is crucial to our develop- 
ment of numerical methods and it is assumed that the reader is at least familar with 
this subject area. Given below is some notation and some of the important relations 
between matrices and vectors. 


A.1 Notation 


1. In the present context a vector is a vertical column or string. Thus 


; -T , ; 
and its transpose v is the horizontal row 
oT > 
u cus E ccs t , v = [v1, V2, V3, . . -, Um 


2. A general m x m matrix А can be written 


ај 012 :^* Aim 
аг Q22 7 Gam 
A = (aij) = A 


Qmi Am2 *** Qmm 


249 
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3. An alternative notation for A is 
A= [a1 2, m 


and its transpose AT is 


APS |e 


4. The inverse of a matrix (if it exists) is written A^! and has the property that 
АТА = АА! = I, where I is the identity matrix. 


A.2 Definitions 


1. A is symmetric if AT = A. 
2. A is skew-symmetric or antisymmetric if AT = — A. 


З. A is diagonally dominant if ay > У 2 |а|, i= 1,2,..., m and ay > X jz lai; 
for at least one i. 


4. A is orthogonal if aj; are real and ATA = AAT = І 

5. A is the complex conjugate of А. 

6. P is a permutation matrix if Pvisa simple reordering of v. 
T. The trace of a matrix is 5; aj. 

8. A is normal if AT A = AAT. 

9. det|A] is the determinant of A. 


10. AP is the conjugate transpose of A, (Hermitian). 








11. If ; 
a 
та с A 
then 
ае А] = ad — bc 
and i Р Ь 
PIS | = 
ае 4] 1-е a 
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A.3 Algebra 


We consider only square matrices of the same dimension. 
1. A and B are equal if а;; = 5j; for all 7,7 = 1,2,...,m. 
2. A-- (B t C) = (C + A) + B, etc. 
3. sA = (saij) where s is a scalar. 
4. In general AB Z BA. 


5. Transpose equalities: 


[ 
d> 
У 
+ 
з 
У 


(4+ В)? 
(AT)? = A 
(AB)! = В?АТ 


6. Inverse equalities (if the inverse exists): 


(41 = A 
(AB) BS AST 
(At = (Arie 


7. Any matrix A can be expressed as the sum of a symmetric and a skew-symmetric 
matrix. Thus: 


1 тү 1 т 
AS (ати) иви) 
A.4  Eigensystems 
1. The eigenvalue problem for a matrix A is defined as 
Ах= А or [A— A]]r 2 0 
and the generalized eigenvalue problem, including the matrix D, as 


Ат = АВт or [A— AB|r =0 


2. If a square matrix with real elements is symmetric, its eigenvalues are all real. 
If it is asymmetric, they are all imaginary. 
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3. Gershgorin's theorem: The eigenvalues of a matrix lie in the complex plane in 
the union of circles having centers located by the diagonals with radii equal to 
the sum of the absolute values of the corresponding off-diagonal row elements. 


4. In general, an т х m matrix A has nz linearly independent eigenvectors with 
nz < m and ny distinct eigenvalues (А;) with ny € nz < m. 


5. А set of eigenvectors is said to be linearly independent if 
а: Б: X Xy , тп К 
for any complex а and b and for all combinations of vectors in the set. 
6. If A posseses m linearly independent eigenvectors then A is diagonalizable, i.e., 
X" AX = А 


where X is a matrix whose columns are the eigenvectors, 


Же Ыл 
and A is the diagonal matrix 
Ar dS eme 0 
= 0 № ` | 
MA & 0 
Dr e Ot Àm 


If A can be diagonalized, its eigenvectors completely span the space, and A is 
said to have a complete eigensystem. 


T. If A has m distinct eigenvalues, then A is always diagonalizable, and with 
each distinct eigenvalue there is one associated eigenvector, and this eigenvector 
cannot be formed from a linear combination of any of the other eigenvectors. 


8. In general, the eigenvalues of a matrix may not be distinct, in which case the 
possibility exists that it cannot be diagonalized. If the eigenvalues of a matrix 
are not distinct, but all of the eigenvectors are linearly independent, the matrix 
is said to be derogatory, but it can still be diagonalized. 


9. If a matrix does not have a complete set of linearly independent eigenvectors, 
it cannot be diagonalized. The eigenvectors of such a matrix cannot span the 
space and the matrix is said to have a defective eigensystem. 


A.4. 


10. 


11. 


12. 


13. 


14. 
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Defective matrices cannot be diagonalized but they can still be put into a com- 
pact form by a similarity transform, S, such that 


postage) 595 Ў 
JE. 
0з. 0 


where there are Ё linearly independent eigenvectors and J; is either a Jordan 
subblock or A;. 


A Jordan submatrix has the form 


Ape 1 0 >- 0 

A wo : 

Җ=|0 0 X 0 
Eo. * 

0 0 0 A 


Use of the transform $ is known as putting А into its Jordan Canonical form. 
А repeated root in a Jordan block is referred to as a defective eigenvalue. For 
each Jordan submatrix with an eigenvalue A; of multiplicity r, there exists one 
eigenvector. The other r — 1 vectors associated with this eigenvalue are referred 
to as principal vectors. The complete set of principal vectors and eigenvectors 
are all linearly independent. 


Note that if P is the permutation matrix 


0 0 1 
Р= |01 0 , pie PRAP 
1 0 0 
then 
A 10 А 


pet А 1|Р=|1 


сз 
о 
> 
© 
= oO 
~ о о 


Some of the Jordan subblocks may have the same eigenvalue. For example, the 
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matrix 


is both defective and derogatory, having: 


e 9 eigenvalues 
e 3 distinct eigenvalues 
e 3 Jordan blocks 


e 5 linearly independent eigenvectors 


З principal vectors with A; 


e | principal vector with A» 


A.5 Vector and Matrix Norms 
1. The spectral radius of a matrix A is symbolized by o(A) such that 
o(A) = |m|max 
where с are the eigenvalues of the matrix A. 
2. A p-norm of the vector V is defined as 
M 1/р 
Ilvll = (> м) 
j=l 


3. A p-norm of a matrix A is defined as 





А||, = max 
| |l» 240 loll, 
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4. Let A and B be square matrices of the same order. All matrix norms must have 
the properties 


IAI] > 0, 141 2 0 implies А = 0 
lle- Al] = |с|-[|А|| 
|А+В|| € ||А|+[|В|| 
[А-В|| < IJAll- |B| 
5. Special p-norms are 
Ali = тах; м У, |а; maximum column sum 
т 
|All2 = с(А - A) 
|[А||» = алома [а] maximum row sum 


where || A||; is referred to as the Lp norm of A. 


6. In general c (A) does not satisfy the conditions in 4, so in general o(A) is not a 
true norm. 


т. When A is normal, o(A) is a true norm, in fact, in this case it is the Ly norm. 


8. The spectral radius of A, o( A), is the lower bound of all the norms of A. 
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Appendix B 


SOME PROPERTIES OF 
TRIDIAGONAL MATRICES 


B.1 Standard Eigensystem for Simple Tridiagonals 


In this work tridiagonal banded matrices are prevalent. It is useful to list some of 
their properties. Many of these can be derived by solving the simple linear difference 
equations that arise in deriving recursion relations. 

Let us consider a simple tridiagonal matrix, i.e., a tridiagonal with constant scalar 
elements a,b, and c, see Section 3.4. If we examine the conditions under which the 
determinant of this matrix is zero, we find (by a recursion exercise) 





det[B(M :a,b,c)]] = 0 
if 
тт 
b+ 2% —— | =0 =1,2,---, M 
+ 2y ac cos (5 ЯЕ -) , m A 
From this it follows at once that the eigenvalues of B(a, b, c) are 
тт 
Е E О BÀ 
+ 2A/ac cos (5 s -) m (B.1) 


The right-hand eigenvector of B(a,b,c) that is associated with the eigenvalue Am 
satisfies the equation 





B(a,b, c), = А, (B.2) 
and is given by 
j—1 
Arne (D) (grt). meiner ma 
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These vectors are the columns of the right-hand eigenvector matrix, the elements of 
which are 





dee f : 
кач | — a 2 ш Jmn. j =1,2,---, M 
а a d been | m=1,2,---,M ae 
Notice that if a = —1 and c= 1, 
ab aa 
(2) Um (B.5) 
c 


The left-hand eigenvector matrix of B(a, b,c) can be written 











— 1 | 
RE E s o 
|». M1 a M+1 | ру М 
In this case notice that if a = —1 and c = 1 
т = 1 
—i(m—1) 1 
(5) d espe с. (B.6) 
a 


B.2 Generalized Eigensystem for Simple Tridiag- 
onals 


This system is defined as follows 








b c Tı e f Tı 
a b c Хә d e f Lo 
a b 33 | =X d e T3 
: : f | 
a b TM d € TM 
In this case one can show after some algebra that 
det[B(a — Ad, b — Ae, c — Af] = 0 (B.7) 
if 
ED E (TREE WETTER. Ло ( | = —1,2,--., M (B.8) 
m m m М +1 5 ? т = ? ? ? 
If we define E 
0,, — Dm, = COS Am, 
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„ = S Xd + а/)рь + 2pm y (ec — fb) (ea — bd) + [(ed — af)paT 


е? — Af dpm 
The right-hand eigenvectors are 
7—1 
э i= Aga 3... [6] m =1,2,---,M 
Tm = |= sin [705], 
€ — А} g j 2 eM 


'These relations are useful in studying relaxation methods. 


B.3 The Inverse of a Simple Tridiagonal 


The inverse of В(а, b,c) can also be written in analytic form. Let D; represent the 
determinant of B(M : a,b, с) 


Dy = det[B(M : a,b, с)| 


Defining Do to be 1, it is simple to derive the first few determinants, thus 


Dy = 1 
Ру = b 

D, = № - ас 

D; = b’ —2abc (В.9) 


One can also find the recursion relation 
Dy m bDy. 4 = QC D уә (B.10) 


Eq. B.10 is a linear OAE the solution of which was discussed in Section 4.2. Its 
characteristic polynomial P(E) is P(E? — bE + ac) and the two roots to P(o) = 0 
result in the solution 
1 b+ Vb? — 4ac ас b — Vb? — Дас in 

2 2 
M 0,1, 9:5 (B.11) 


where we have made use of the initial conditions Dy = 1 and D, = b. In the limiting 
case when b? — 4ac = 0, one can show that 


b M 
Du = (М +1)(3) ; b = Ade 


260 
Then for M = 4 
Da —cD» eD Se Do 
gel — 1 —aD» Dı Do —cD,D, е? Р, 
D, a? D, —a Dı Dı DoD, —cDo 
—а%Г% а? Р, —a D» Da 
and for М = 5 
D, —c D; е? Р» -eD e Do 
—aDa D4Da —cD4D» cD,Dj ep D 
c Р» 


1 
D. a? D» —aDı Də 7р» —cD34D4 
? 0° D; a? Dı D, —a DD; D, Dı —c D} 
at Do —a®D, a? D» —a.Ds D, 


B~ = 


The general element dmn is 


Upper triangle: 
т = 1,2, -- M—1 ; n-m-cl,m42,--,M 


dmn = Da iDyu-s(-c) ™/Dm 


п = т = 1,2,---,М 
dmm ма Риу. Du 


Lower triangle: 


т=п+1,п+2,:-:,М ; n=1,2,---,;M-1 


dmn = Du-mDn-1(—a)™ "/Du 


B.4  Eigensystems of Circulant Matrices 


B.4.1 Standard Tridiagonals 


Consider the circulant (see Section 3.4.4) tridiagonal matrix 


B,(M : a,b,c, ) 
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(B.12) 
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The eigenvalues are 


2 2 
Am = 6+(a+c)cos (=>) — i(a — с) sin (=>) , m-0,,2,--.,M-1 
(B.13) 
The right-hand eigenvector that satisfies B,(a, b, C) m = \mim is 
Lm = (Zj)m eu umo И j =0,1,---,M — 1 (В.14) 


where 7 = /—1, and the right-hand eigenvector matrix has the form 


The left-hand eigenvector matrix with elements 2’ is 


с") 

X` = (Emi) = мМ 

Note that both X and X^! are symmetric and that X^! = x", where X* is the 
conjugate transpose of X. 


B.4.2 General Circulant Systems 


Notice the remarkable fact that the elements of the eigenvector matrices X and X^! 
for the tridiagonal circulant matrix given by eq. B.12 do not depend on the elements 
a,b,c in the matrix. In fact, all circulant matrices of order M have the same set of 
linearly independent eigenvectors, even if they are completely dense. An example of 
a dense circulant matrix of order M — 4 is 


bo bı 09 bs 
bs bg bi b 
С b, be 1 (B.15) 
bi bz ba bo 


The eigenvectors are always given by eq. B.14, and further examination shows that 
the elements in these eigenvectors correspond to the elements in a complex harmonic 
analysis or complex discrete Fourier series. 

Although the eigenvectors of a circulant matrix are independent of its elements, 
the eigenvalues are not. For the element indexing shown in eq. B.15 they have the 
general form 


p = е 


of which eq. B.13 is a special case. 
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B.5 Special Cases Found From Symmetries 


Consider a mesh with an even number of interior points such as that shown in Fig. 
B.1. One can seek from the tridiagonal matrix B(2M : а, Б,а, ) the eigenvector subset 
that has even symmetry when spanning the interval 0 < x < л. For example, we seek 
the set of eigenvectors Tm for which 


b a Tı 21 
а b a To 19 
а d. 

а 
a b а||х» Lo 
a bl |z 11 


This leads to the subsystem of order M which has the form 


b a 
a b a 
B(M : a,b, а) = NM Im = Ann (B.16) 
а 
a b a 
a b+a 


By folding the known eigenvectors of B(2M : a,b,a) about the center, one can show 
from previous results that the eigenvalues of eq. B.16 are 


(2m — 1)т 
m = b + 2a cos | S — 
Л b+ acos ( 2M 1 


) , m=1,2,---,M (B.17) 
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and the corresponding eigenvectors are 


>. fj(2m-l)r 
Ege SR (л | 


7 = 1,2,---, M 


Line of Symmetry 


Imposing symmetry about the same interval 
but for a mesh with an odd number of points, 
see Fig. B.1, leads to the matrix 


a. An even-numbered mesh 


b a 
a b a Line of Symmetry 
B(M :а,Ь,а) = á 
а 
a b а 
2a b 


By folding the known eigenvalues of B(2M — 
1: a,b,a) about the center, one can show 
from previous results that the eigenvalues of b. An odd-numbered mesh 
eq. В.17 are 


Figure B.1 — Symmetrical folds for 
special cases 





2m — 1 
Am = b + 2a cos СТЕ 


= ) ; m=1,2,--,M 


and the corresponding eigenvectors are 


Е | (= 1)т 


= = 1,2,---,M 
Lm = SIN 2M | > J › 2; , 


B.6 Special Cases Involving Boundary Conditions 


We consider two special cases for the matrix operator representing the 3-point central 
difference approximation for the second derivative 02/01? at all points away from the 
boundaries, combined with special conditions imposed at the boundaries. 
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Note: In both cases 





When the boundary conditions are Dirichlet on both sides, 








He. n 
1-2 1 x = -2 + 2cos ( ПЕ ) 
ПЕ sw i» rre ALI (B.18) 
1—2 1 Tm = sin (x E) 
[ <9 


When one boundary condition is Dirichlet and the other is Neumann (and a diagonal 
preconditioner is applied to scale the last equation), 


—2 1 
Dac 1 Am = —2+2cos Gn я 
iu (B.19) 


Appendix C 


THE HOMOGENEOUS 
PROPERTY OF THE EULER 
EQUATIONS 


The Euler equations have a special property that is sometimes useful in constructing 
numerical methods. In order to examine this property, let us first inspect Euler's 
theorem on homogeneous functions. Consider first the scalar case. If F(u, v) satisfies 
the identity 

F(au, ov) = o" F(u, v) (C.1) 


for a fixed n, F is called homogeneous of degree n. Differentiating both sides with 
respect to a and setting o = 1 (since the identity holds for all a), we find 


u— +v— = пЕ(и, 0) (С.2) 


Consider next the theorem as it applies to systems of equations. If the vector 
F(Q) satisfies the identity 


F(oQ) = o" F(Q) (C.3) 


for a fixed n, F is said to be homogeneous of degree n and we find 


|o. |e - nr) (са) 
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Now it is easy to show, by direct use of eq. C.3, that both E and F in eqs. 2.11 and 
2.12 are homogeneous of degree 1, and their Jacobians, A and B, are homogeneous 
of degree 0 (actually the latter is a direct consequence of the former).' This being 
the case, we notice that the expansion of the flux vector in the vicinity of tn which, 
according to eq. 6.105 can be written in general as, 


х En T АЛО = Qn) F O(h?) 
F = Fa + Ba(Q- Qn) +O(h’) (C.5) 


can be written 


E = A,Q + O(h’) 
F = B,Q + O(h?) (C.6) 


since the terms En — AnQn and Fp — B4,Q, are identically zero for homogeneous 
vectors of degree 1, see eq. C.4. Notice also that, under this condition, the constant 
term drops out of eq. 6.106. 

As a final remark, we notice from the chain rule that for any vectors F and Q 





OF (Q) OF | 0Q OQ 
= || Д z 
Ox РА Ox Ox oe 
We notice also that for a homogeneous F of degree 1, F = AQ and 
OF OQ OA 
Ge ar bac oy 


Therefore, if F is homogeneous of degree 1, 


дА 

— |Q =0 C.9 
bac (C9) 
in spite of the fact that individually [OA/Ox] and Q are not equal to zero. 


‘Note that this depends on the form of the equation of state. The Euler equations are homoge- 
neous if the equation of state can be written in the form p = pf(e), where є is the internal energy 
per unit mass. 
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